MaxDiff scores

After collecting responses to your MaxDiff exercise, MaxDiff scores (importance scores) are estimated using Hierarchical Bayesian (HB) utility estimation. These scores provide powerful insights by ranking items and revealing their relative importance or preference.

Compared to traditional ranking questions, MaxDiff scores offer greater discrimination and avoid the scale use bias often seen in rating scales.

MaxDiff functions like a "beauty contest" among items. As respondents choose their most and least preferred items, Discover calculates preference scores for each item.

Each item receives a positive, ratio-scaled score that sums to 100. This scaling indicates that an item with a score of 10 is twice as preferred or important as one with a score of 5.

You can view these scores in the Summary tab of the exercise analysis area. Results are often visualized by listing items from highest to lowest scores for the overall sample or for specific respondent subgroups.

Max Diff Scores Summary Charts in Discover

For additional insight, you can display 95% confidence intervals alongside MaxDiff scores.

Assuming your respondents are representative of the randomly drawn population, you can be 95% confident that the true population score falls within this interval.

You can also use the 95% confidence intervals to determine if one item is preferred over another. If the intervals for two items do not overlap, you can be at least 95% confident that one item is preferred to the other.

Segmentation enables comparisons of MaxDiff results among respondent groups based on survey questions or variables.

For example, if your MaxDiff exercise compares music artists, you can segment the results by the respondents' locations (asked in another question) to compare preferences between North America and Europe.

To apply segmentation:

Click the Segmentation dropdown.
Select a question or variable from the menu.
The segmentation icon will turn green to confirm it's active.

To remove segmentation or apply a different one:

Click the Segmentation dropdown.
Select No segmentation or choose another option.

Segmenting Dropdown in Max Diff Analysis

When segmentation is active, the results will update to show scores for each group. Respondents who do not have a defined value for the selected variable will be excluded from the segmented results.

In the upper right corner of the settings panel, you'll see the download menu (). This menu provides five available file downloads:

Scores (.xlsx)
Individual Scores-Rescaled (.xlsx)
Individual Scores-Raw (.xlsx)
Counts (.xlsx)
Charts (.png)

Each download offers useful insights for analyzing your MaxDiff results.

The Scores download contains a summary of the MaxDiff Utility scores (identical to the table viewable in Discover). Scores are rescaled.

Individual MaxDiff scores can be downloaded in two formats: Raw and Rescaled.

In both downloads, each respondent's data includes a column labeled MaxDiff_Fit (RLH). This fit statistic utilizes root likelihood (hence "RLH") to indicate the probability that each respondent made their choices based on their preference or utility scores. RLH can be understood as the geometric mean of the probabilities that the raw utilities explain the respondent's choices.

Raw scores represent the regression weights from a multinomial logistic regression model typically utilizing a Hierarchical Bayes (HB) MNL model. These scores are centered around zero, meaning their average value is 0, unless anchored scaling is applied in MaxDiff. In anchored scaling, scores are positive for items preferred more than the anchor point and negative for those less preferred.

Although not as intuitive as rescaled scores, raw scores are valuable for advanced researchers aiming to predict choice probabilities among utilities with the logit equation.

Rescaled scores are always positive, sum to 100, and follow a ratio scale. A score of 10 means the item is twice as preferred as an item with a score of 5.

Rescaled scores are more intuitive for most audiences to understand. For details on how the raw HB scores are converted to probabilities that add up to 100, see Appendix K of the  CBC/HB manual,  specifically the section titled "A Suggested Rescaling Procedure."

The Counts download provides detailed information on how often each item appeared in the MaxDiff exercise and how respondents reacted to those items. The data includes:

Number of times each item was shown
Number of times each item was selected as “best” or “worst”
Count proportions, which express the likelihood that an item was chosen when shown

Count proportions indicate the likelihood of an item being selected when presented. The data includes columns that report these probabilities:

Best count probability — The likelihood that respondents selected the item as “best” when it appeared in a MaxDiff set.
Worst count probability — The likelihood that respondents selected the item as “worst.”

Since these are probabilities, count proportions are restricted to values between 0 and 1. For example, if four items are shown in each task, the probability of selecting any given item is 25%. Therefore, an item with a "best" count proportion of 50% is considered the "best” at twice the chance level (assuming four items per task).

To obtain a quick summary score that is highly correlated with Hierarchical Bayes (HB) results, subtract an item's "worst" count proportion from its "best" count proportion.

The Charts download exports a .png image identical to the Discover Scores chart. This option is useful for quickly sharing or presenting MaxDiff results without having to recreate the visual in other software.

Discover Help Guide

MaxDiff scores

Introduction

Interpreting scores

Confidence intervals

Segmenting

Downloads

Scores

Individual MaxDiff scores

Raw scores

Rescaled scores

Counts

Count proportions

Charts