Sawtooth Software Conference 2019 | San Diego, CA, USA | September 23–27
 

A Discrete Choice Take on Halloween: Results

As promised in my fun Halloween article, we have the results!  With a significantly smaller network than @fivethirtyeight, our sample size is on the lower end at n=73. That being said - MaxDiff does an amazing job with small sample sizes and the data proves it!  Comparing our n=73 results to the n=8,300+ of @fivethirtyeight, we arrive at the same top 5 candies. Now we just have to argue over who is right!

Just a quick reminder of the MaxDiff exercise.  There were 80 total items and respondents saw a subset of 5 items per screen, across 16 screens.  This meant that each person saw every item once. On each screen, the respondent was asked to imagine they were trick-or-treating for Halloween and, given these five options, which was the best and which was the worst to receive.   

Alright, let’s take a look at the results!

For the top 5 items out of 80, our n=73 completes yielded the same 5 items as @FiveThirtyEight.  While the @FiveThirtyEight followers seem to be keen on Peanut Butter – our followers are more into the “crunch” with Twix taking the number one spot followed by Kit Kat.

  Sawtooth Software’s MaxDiff FiveThirtyEight’s Paired Comparisons
  Rank Probability Rank Probability
Twix 1 88% 3 82%
Kit Kat 2 84% 4 77%
Reese’s Peanut Butter Cup 3 81% 1 84%
Snickers 4 78% 5 77%
Reese’s Minis 5 72% 2 82%

In our survey, we included what’s known as an anchoring question - Here is a subset of items based on your answers, now select all the items that you would actually want to receive as a trick-or-treater.  We can use each individual’s answers to this anchor question to draw a line in the sand and say that the items that appear above the anchor are items people actually want to receive and the items below the anchor, they do not.  

The probability scores shown in the table below range from 0 to 100 and reflect the likelihood that an item would be selected as "best" compared to the Anchor.  A score of 50% indicates the item is equal in utility to the Anchor item. A score of 90% means the item has a 90% likelihood of being selected instead of the Anchor item.  A score of 10% means the item has a 10% likelihood of being selected instead of the Anchor item.

One thing I personally found interesting was where the monetary items appear in the list (one dollar, one quarter, and one dime).  Our results show that there are seven items that have a higher utility than the dollar! And those items definitely don’t cost $1 so there is something about the Halloween holiday that makes people forget that they are price sensitivity.

Item Score Rank
Twix 88.16% 1
Kit Kat 83.83% 2
Reese's Peanut Butter Cup 80.93% 3
Snickers 78.37% 4
Reeses Minis 71.55% 5
Butterfinger 66.82% 6
Milky Way 63.33% 7
Reeses Pieces 63.30% 8
One Dollar 62.95% 9
Sour Patch Kids 62.34% 10
Snickers Crisper 59.32% 11
Milky Way Simply Caramel 57.69% 12
Rolo 55.70% 13
100 Grand 54.34% 14
M&Ms Peanut 53.82% 15
Snickers Peanut Butter 53.35% 16
Hershey's Krackel 51.90% 17
Almond Joy 51.08% 18
Snickers Almond 50.48% 19
Milky Way Midnight 49.71% 20
Skittles 47.65% 21
Junior Mints 46.79% 22
Heath Bar 46.37% 23
Baby Ruth 46.02% 24
Nestle Crunch 45.77% 25
Starburst 45.62% 26
Haribo Gold Bears 45.26% 27
Mr Goodbar 44.85% 28
3 Musketeers 44.52% 29
Lifesavers Big Ring Gummies 44.52% 30
Airheads 44.34% 31
Swedish Fish 43.38% 32
Warheads Sour Chewy Cubes 42.99% 33
Mounds 42.95% 34
Hershey's Special Dark 37.98% 35
Laffy Taffy 37.55% 36
Nerds 37.51% 37
Warheads Sour Hard Candy 35.67% 38
Payday 35.13% 39
Milk Duds 34.46% 40
Mike & Ike 34.10% 41
M&Ms 33.58% 42
Chewy Lemonhead Fruit Mix 33.26% 43
Twizzlers 33.08% 44
Pop Rocks 32.48% 45
Hershey's Milk Chocolate 32.34% 46
One Quarter 31.92% 47
Charleston Chew 29.00% 48
Dots 26.04% 49
Blow Pop 25.73% 50
Hershey's Kisses 25.48% 51
Werthers Original 24.19% 52
Now And Later 23.29% 53
Fruit Chews 23.07% 54
Lemonhead 23.06% 55
Whoppers 22.62% 56
Gobstopper 22.49% 57
Fun Dip 22.24% 58
Runts 21.92% 59
Ring Pop 20.66% 60
Tootsie Pop 19.93% 61
Red Vines 19.79% 62
Tootsie Roll Juniors 19.49% 63
Tootsie Roll Midgees 16.51% 64
Pixy Stix 16.06% 65
Caramel Apple Pops 15.79% 66
Sixlets 13.85% 67
Root Beer Barrels 13.72% 68
Dum Dums 12.83% 69
Nik-L-Nip 12.33% 70
Strawberry Bon Bons 10.97% 71
Candy Corn 10.59% 72
One Dime 10.03% 73
Smarties 9.78% 74
Sugar Daddy 9.52% 75
Chiclets 9.22% 76
Sugar Babies 8.68% 77
Boston Baked Beans 6.48% 78
Jaw Busters 5.07% 79
Super Bubble 4.97% 80

Another great thing we can do with MaxDiff data is run a cluster analysis.  This helps us to make sure that we aren’t overlooking the heterogeneity that exists between humans (i.e. we each like different candy options!).  With only N=73 completes, it’s tough to draw hard conclusions, but directionally we can see two distinct segments appear. The first aligns with our top 5 ranks – let’s call them the “Halloween Basics”.  The second segment is “Searching for Sour” with Sour Patch Kids, AirHeads and Warheads earning the top 3 ranks. Which segment are you? Let us know in the comments!

Finally, we can take this data into a TURF simulator and figure out how we can make the entire neighborhood happy.  Given what we know about the overall ranks, and our newly found clusters it should be no surprise that we can “reach” almost the entire neighborhood with a bundle of:  Twix, Sour Patch Kids and Reese's Peanut Butter Cups.

Not a fan of Sour Patch Kids? Sub them out for Warheads and reach almost the same amount.  Allergic to Peanut Butter? Trade those in for Kit Kats and reach almost the same amount too.

The data doesn’t lie – people love candy on Halloween!  So next Halloween, create a MaxDiff survey for your neighborhood and be the favorite house on the block.