CBC design settings

Behind the scenes, the Discover exercise designer selects attribute levels to show across multiple tasks (questions) according to the following goals:

• Each level within the same attribute should appear an equal number of times.
• Each level of an attribute should appear with levels of other attributes an equal number of times.
• As a final step, the designer does some rearranging of the concept order within each task and rearranging the task order to try to improve:
  • The balance in how often each level appears in each concept position (left, middle, right, etc.).
  • How evenly dispersed attribute levels are across tasks, trying to avoid showing the same level in consecutive (or nearby) tasks.

Depending on your attribute/level list and number of tasks (questions) to show, it may not be possible to achieve perfection on all three goals; but CBC questionnaires that are almost-but-not-quite perfect are still very efficient and you will obtain excellent results in practice.

The Number of CBC tasks setting configures the number of tasks (individual questions within the exercise) that the respondent must answer.

It is possible that Discover will recommend a design that has a high CBC task count. In general, we don’t recommend that you decrease the Number of CBC tasks because this reduces the information available to estimate accurate preference scores.

If the recommended range for Number of CBC tasks exceeds what you think respondents can answer without getting tired and giving you bad data, then you should consider the following:

• The best course of action to remedy this situation is to reduce the number of attributes or number of levels per attribute (therefore reducing the need for additional tasks) in your CBC exercise. For the best experience (limited respondent fatigue and better data quality), we recommend that CBC exercises generally don’t have more than about 10 attributes.
• If you have a relatively large sample size (often in the multiple 100s of respondents), then you can count on individual errors tending to cancel out due to the benefit of large numbers of respondents. As your sample size becomes very large, you may decide you can afford to reduce the accuracy at the individual level (by reducing the number of choice tasks) due to the abundance of respondent records.

Based on the largest number of levels in any one of your attributes, the Discover designer recommends the Number of concepts per task (excluding the "None" concept).

From a statistical viewpoint, showing more Concepts per task is better for obtaining precise preference scores. However, showing more concepts per task makes it harder for humans to see the information on smaller screens and accurately complete the CBC exercise.

We generally suggest presenting at least three Concepts per task (in addition to the none option). However, in certain cases such as emotionally laden decisions involving different cancer treatments, it may be extremely difficult for respondents to consider more than two concepts at a time.

If you have provided a lot of attributes or levels, the designer may suggest a Number of CBC Tasks that is difficult for respondents to adequately complete. Although the recommendation may be high, the warning message is still shown because that task count is needed to produce accurate utility scores for each respondent. Consider reducing the number of attributes and levels to bring down the recommendation.

If you have ample sample size and are willing to accept more noise at the individual level than is typical for conjoint analysis, you may proceed with a sparse design featuring fewer tasks than recommended.

Since 30 tasks is likely difficult for a respondent to accurately complete, the software gives a warning when you adjust the design to require more than this. You may choose to still use this design but proceed with caution.

At least one task must be shown to respondents to field a CBC exercise.

It may be confusing to respondents if there are concepts that look the exact same in a single task. The software alerts you when you will no longer be able to have completely unique concepts in the same task.

At least two concepts (aside from the None) must be shown to the respondents in Discover.

Requiring respondents to compare many concepts may be taxing and could lead to poor utility scores. It is also difficult to view on small screens.

A CBC exercise cannot include more than 100 concepts in a single task.

A CBC exercise cannot include more than 100 tasks.