As promised in my fun Halloween article, we have the results! With a significantly smaller network than @fivethirtyeight, our sample size is on the lower end at n=73. That being said - MaxDiff does an amazing job with small sample sizes and the data proves it! Comparing our n=73 results to the n=8,300+ of @fivethirtyeight, we arrive at the same top 5 candies. Now we just have to argue over who is right!
Just a quick reminder of the MaxDiff exercise. There were 80 total items and respondents saw a subset of 5 items per screen, across 16 screens. This meant that each person saw every item once. On each screen, the respondent was asked to imagine they were trick-or-treating for Halloween and, given these five options, which was the best and which was the worst to receive.
Alright, let’s take a look at the results!
For the top 5 items out of 80, our n=73 completes yielded the same 5 items as @FiveThirtyEight. While the @FiveThirtyEight followers seem to be keen on Peanut Butter – our followers are more into the “crunch” with Twix taking the number one spot followed by Kit Kat.
| Sawtooth Software’s MaxDiff | FiveThirtyEight’s Paired Comparisons | |||
|---|---|---|---|---|
| Rank | Probability | Rank | Probability | |
| Twix | 1 | 88% | 3 | 82% |
| Kit Kat | 2 | 84% | 4 | 77% |
| Reese’s Peanut Butter Cup | 3 | 81% | 1 | 84% |
| Snickers | 4 | 78% | 5 | 77% |
| Reese’s Minis | 5 | 72% | 2 | 82% |
In our survey, we included what’s known as an anchoring question - Here is a subset of items based on your answers, now select all the items that you would actually want to receive as a trick-or-treater. We can use each individual’s answers to this anchor question to draw a line in the sand and say that the items that appear above the anchor are items people actually want to receive and the items below the anchor, they do not.
The probability scores shown in the table below range from 0 to 100 and reflect the likelihood that an item would be selected as "best" compared to the Anchor. A score of 50% indicates the item is equal in utility to the Anchor item. A score of 90% means the item has a 90% likelihood of being selected instead of the Anchor item. A score of 10% means the item has a 10% likelihood of being selected instead of the Anchor item.
One thing I personally found interesting was where the monetary items appear in the list (one dollar, one quarter, and one dime). Our results show that there are seven items that have a higher utility than the dollar! And those items definitely don’t cost $1 so there is something about the Halloween holiday that makes people forget that they are price sensitivity.
| Item | Score | Rank |
|---|---|---|
| Twix | 88.16% | 1 |
| Kit Kat | 83.83% | 2 |
| Reese's Peanut Butter Cup | 80.93% | 3 |
| Snickers | 78.37% | 4 |
| Reeses Minis | 71.55% | 5 |
| Butterfinger | 66.82% | 6 |
| Milky Way | 63.33% | 7 |
| Reeses Pieces | 63.30% | 8 |
| One Dollar | 62.95% | 9 |
| Sour Patch Kids | 62.34% | 10 |
| Snickers Crisper | 59.32% | 11 |
| Milky Way Simply Caramel | 57.69% | 12 |
| Rolo | 55.70% | 13 |
| 100 Grand | 54.34% | 14 |
| M&Ms Peanut | 53.82% | 15 |
| Snickers Peanut Butter | 53.35% | 16 |
| Hershey's Krackel | 51.90% | 17 |
| Almond Joy | 51.08% | 18 |
| Snickers Almond | 50.48% | 19 |
| Milky Way Midnight | 49.71% | 20 |
| Skittles | 47.65% | 21 |
| Junior Mints | 46.79% | 22 |
| Heath Bar | 46.37% | 23 |
| Baby Ruth | 46.02% | 24 |
| Nestle Crunch | 45.77% | 25 |
| Starburst | 45.62% | 26 |
| Haribo Gold Bears | 45.26% | 27 |
| Mr Goodbar | 44.85% | 28 |
| 3 Musketeers | 44.52% | 29 |
| Lifesavers Big Ring Gummies | 44.52% | 30 |
| Airheads | 44.34% | 31 |
| Swedish Fish | 43.38% | 32 |
| Warheads Sour Chewy Cubes | 42.99% | 33 |
| Mounds | 42.95% | 34 |
| Hershey's Special Dark | 37.98% | 35 |
| Laffy Taffy | 37.55% | 36 |
| Nerds | 37.51% | 37 |
| Warheads Sour Hard Candy | 35.67% | 38 |
| Payday | 35.13% | 39 |
| Milk Duds | 34.46% | 40 |
| Mike & Ike | 34.10% | 41 |
| M&Ms | 33.58% | 42 |
| Chewy Lemonhead Fruit Mix | 33.26% | 43 |
| Twizzlers | 33.08% | 44 |
| Pop Rocks | 32.48% | 45 |
| Hershey's Milk Chocolate | 32.34% | 46 |
| One Quarter | 31.92% | 47 |
| Charleston Chew | 29.00% | 48 |
| Dots | 26.04% | 49 |
| Blow Pop | 25.73% | 50 |
| Hershey's Kisses | 25.48% | 51 |
| Werthers Original | 24.19% | 52 |
| Now And Later | 23.29% | 53 |
| Fruit Chews | 23.07% | 54 |
| Lemonhead | 23.06% | 55 |
| Whoppers | 22.62% | 56 |
| Gobstopper | 22.49% | 57 |
| Fun Dip | 22.24% | 58 |
| Runts | 21.92% | 59 |
| Ring Pop | 20.66% | 60 |
| Tootsie Pop | 19.93% | 61 |
| Red Vines | 19.79% | 62 |
| Tootsie Roll Juniors | 19.49% | 63 |
| Tootsie Roll Midgees | 16.51% | 64 |
| Pixy Stix | 16.06% | 65 |
| Caramel Apple Pops | 15.79% | 66 |
| Sixlets | 13.85% | 67 |
| Root Beer Barrels | 13.72% | 68 |
| Dum Dums | 12.83% | 69 |
| Nik-L-Nip | 12.33% | 70 |
| Strawberry Bon Bons | 10.97% | 71 |
| Candy Corn | 10.59% | 72 |
| One Dime | 10.03% | 73 |
| Smarties | 9.78% | 74 |
| Sugar Daddy | 9.52% | 75 |
| Chiclets | 9.22% | 76 |
| Sugar Babies | 8.68% | 77 |
| Boston Baked Beans | 6.48% | 78 |
| Jaw Busters | 5.07% | 79 |
| Super Bubble | 4.97% | 80 |
Another great thing we can do with MaxDiff data is run a cluster analysis. This helps us to make sure that we aren’t overlooking the heterogeneity that exists between humans (i.e. we each like different candy options!). With only N=73 completes, it’s tough to draw hard conclusions, but directionally we can see two distinct segments appear. The first aligns with our top 5 ranks – let’s call them the “Halloween Basics”. The second segment is “Searching for Sour” with Sour Patch Kids, AirHeads and Warheads earning the top 3 ranks. Which segment are you? Let us know in the comments!
Finally, we can take this data into a TURF simulator and figure out how we can make the entire neighborhood happy. Given what we know about the overall ranks, and our newly found clusters it should be no surprise that we can “reach” almost the entire neighborhood with a bundle of: Twix, Sour Patch Kids and Reese's Peanut Butter Cups.
Not a fan of Sour Patch Kids? Sub them out for Warheads and reach almost the same amount. Allergic to Peanut Butter? Trade those in for Kit Kats and reach almost the same amount too.
The data doesn’t lie – people love candy on Halloween! So next Halloween, create a MaxDiff survey for your neighborhood and be the favorite house on the block.