MaxDiff Methodology at Blizzard: Insights and Applications

MaxDiff (Maximum Difference Scaling) is a powerful research methodology often used to measure preferences and priorities among a set of items. In a recent webinar hosted by Sawtooth Software, Hasan Elsadig, Global Research and Consumer Insights Manager at Blizzard Entertainment, shared his experience with applying MaxDiff methodology in the gaming industry. This blog post summarizes key insights from his presentation, explores practical applications of MaxDiff at Blizzard, and highlights relevant expertise on MaxDiff from Sawtooth Software.

Introduction to MaxDiff and Its Relevance to Gaming 

MaxDiff is particularly useful in situations where traditional survey methods, such as rating scales or ranking questions, fail to provide clear differentiation among items. This is often the case in the gaming industry, where gamers' preferences can be highly subjective and such respondents can become lazy/unengaged about expressing their preferences. MaxDiff forces respondents to make trade-offs, revealing more precise priorities. This makes it an ideal tool for understanding the nuanced preferences of gamers, who are diverse, passionate, and yet don’t always engage well with standard ratings-based questions to register their opinions.

Why Blizzard Uses MaxDiff 

At Blizzard, MaxDiff is employed to understand player preferences more accurately. Traditional survey methods can result in flat or biased responses. For instance, gamers might rate all features as important, making it challenging to prioritize development efforts. Other gamers may be biased to use just low points on the scale. MaxDiff addresses this issue by requiring respondents to choose the most and least important items from a set, providing clearer differentiation and more actionable insights—while escaping scale use bias.

How MaxDiff Is Used at Blizzard 

Survey Research Support 

Blizzard uses surveys to gather insights to assist various teams, including design, development, marketing, and product management. These surveys help in understanding player sentiment, feature preferences, and pricing strategies. Some key areas of research include:

• Sentiment Research: Understanding how players feel about the games they play. Are the games fun, rewarding, and worth recommending to others?
• Product and Feature Research: Evaluating new features and mechanics to see if players understand and are interested in them.
• Pricing Research: Determining if the pricing of in-game items and bundles is perceived as fair and provides value to players.

Internal vs. External Research 

Blizzard conducts a mix of internal research, using historical software products and Sawtooth Software's solutions, and external research with various research partners. Although multiple software platforms are used at Blizzard, they are discovering that Sawtooth is able to handle standard surveys that don’t even involve advanced methodologies like MaxDiff.

MaxDiff for Bundle Evaluation 

One practical application of MaxDiff at Blizzard is evaluating in-game cosmetic bundles. By using images of bundles and visual representations of prices, Blizzard can assess the desirability and perceived value of different bundles. This helps in making informed decisions about pricing and product offerings without the need for more complex methodologies like conjoint analysis.

Enhancing MaxDiff Results 

Simplifying Presentation 

To communicate MaxDiff results effectively to various stakeholders, Blizzard often presents findings in a simple, tiered format. This approach makes it easier for non-research-savvy stakeholders to understand and act on the insights. For instance, results might be categorized into tiers that show the relative importance of different features, making it clear which ones should be prioritized.

Using Images in MaxDiff 

Blizzard has experimented with using images instead of text in MaxDiff studies. This is particularly effective in creative industries like gaming, where visual elements are crucial. For example, by using images of in-game items, Blizzard can better capture player preferences and reduce cognitive load on respondents. This approach helps in making more accurate and actionable decisions based on player preferences.

Elevating Actionability with Quadrant Maps 

Quadrant maps are a valuable tool for enhancing the actionability of MaxDiff results. By combining MaxDiff data with other metrics like satisfaction scores, Blizzard can create 2-dimension quadrant maps that highlight areas of high importance and low satisfaction. This helps prioritize areas for improvement and align research insights with strategic goals.

Performance-Oriented Research 

Combining MaxDiff results with satisfaction scores helps identify features that are both important and underperforming. This allows for targeted improvements that can significantly enhance player experience. For instance, features that fall into the high importance but low satisfaction quadrant are prioritized for development and improvement.

Stated vs. Derived Appeal 

By comparing stated appeal (what players say they want) with derived appeal (what players actually prefer), Blizzard can identify features that fulfill unconscious needs. This approach helps balance giving players what they say they want and what they truly need for a better gaming experience. For example, features with high derived appeal but low stated appeal might represent hidden needs that can provide a competitive advantage if addressed.

Expertise on MaxDiff from Sawtooth Software 

Sawtooth Software is the leader in the field of MaxDiff and conjoint analysis, providing powerful tools and methodologies for market research. Here are some key insights and best practices from Sawtooth Software's expertise in MaxDiff:

Flexibility and Customization 

Sawtooth Software's tools, such as Discover and Lighthouse Studio, offer flexibility in designing MaxDiff studies. Researchers can customize surveys to include images, adjust the number of items per set, and tailor the questions to specific research needs. This flexibility is crucial for industries like gaming, where visual and contextual elements are significant.

Handling Complex Methodologies 

While MaxDiff is effective for straightforward preference measurement, Sawtooth Software also provides solutions for more complex methodologies like conjoint analysis (also known as discrete choice modeling). These methodologies are useful when researchers need to understand the trade-offs between different attributes and levels, such as in product bundling or pricing studies.

Advanced Analysis and Reporting 

Sawtooth Software offers advanced analysis and reporting capabilities, allowing researchers to derive actionable insights from MaxDiff data. This includes tools for segmenting respondents, creating quadrant maps, and conducting market simulations. These features help researchers translate data into strategic decisions and practical recommendations.

Training and Support 

Sawtooth Software provides extensive training and support for researchers new to MaxDiff or looking to deepen their expertise. This includes webinars, white papers, and personalized support from experienced professionals. By leveraging these resources, researchers can maximize the value of their MaxDiff studies and apply best practices in their research projects.

Practical Examples from Blizzard 

Using MaxDiff to Understand Player Preferences 

One of the key uses of MaxDiff at Blizzard is to understand player preferences for in-game features and content. For example, when evaluating new features for a game like Diablo IV, Blizzard might use MaxDiff to determine which features are most important to players. This helps prioritize development efforts and ensures that the game meets player expectations.

Evaluating In-Game Bundles 

Blizzard uses MaxDiff to evaluate the appeal of different in-game bundles. By presenting players with images of bundles and asking them to choose the most and least appealing ones, Blizzard can gain insights into which bundles are perceived as the best value. This information is used to optimize pricing and product offerings.

Enhancing Player Experience 

MaxDiff is also used to enhance the overall player experience by identifying areas for improvement. By combining MaxDiff results with satisfaction scores, Blizzard can pinpoint features that are important to players but currently underperforming. This helps focus development efforts on areas that will have the greatest impact on player satisfaction.

Conclusion 

MaxDiff is a versatile and powerful tool that has helped Blizzard better understand and serve its players. By using MaxDiff in innovative ways, such as incorporating images and creating quadrant maps, Blizzard can derive actionable insights that drive game development and player satisfaction. With the support and expertise of Sawtooth Software, Blizzard continues to innovate and enhance its games, ensuring a rewarding experience for players worldwide.

This blog post highlights key insights from Hasan Elsadig's presentation on MaxDiff methodology at Blizzard. By leveraging MaxDiff and other advanced research techniques, Blizzard can stay attuned to player preferences and deliver exceptional gaming experiences. As the gaming industry continues to evolve, methodologies like MaxDiff will play a crucial role in understanding and meeting the needs of players.