At Microsoft Hardware, we have fielded several ACBC studies for consumer PC accessory product lines in the US and Japan, including two studies where we directly compared CBC and ACBC results given identical attributes and features. In our comparative studies, ACBC part worth estimates have been consistent with CBC estimates. At an aggregate level, we've seen improved stability of market simulation estimates across respondent samples using data from ACBC estimates, as compared to CBC. We feel that ACBC estimates are of high quality and are comparable to those we obtain from traditional CBC.
The most important strength of ACBC is that it allows study designs that are difficult or impossible for CBC. ACBC allows direct examination of features that are "must haves" or "won't accepts", which are of immediate interest to product management. Because ACBC drills into preference within each subject's response pattern, it gives us confidence that the entire feature space of interest to respondents is investigated in depth. We are currently exploring the ability of ACBC to work with real-time changes in attribute lists, such as including or removing features from the tasks for certain respondents. This allows experimental manipulation of choice tasks based on prior information in the survey, a capability that is impossible in CBC designs.
In respondent feedback, ACBC has been preferred over CBC even with longer survey times. Respondents rate ACBC trial blocks as less boring than CBC. They respond particularly well to the dialogue format when ACBC asks about their response patterns; verbatim comments have been that ACBC is more "interactive", "fun", and that it "learns" from their responses. We've even seen respondents in a lab setting inquire whether the ACBC adaptive system is controlled by an actual observer behind the scenes. The higher respondent engagement means that complex choice-based surveys are more tolerable to respondents.
Overall, we are increasingly using ACBC alongside traditional CBC and ACA surveys. Each method has advantages for different kinds of product questions and study designs. We gain increased confidence in our models when we combine data from multiple methods, ensuring that we sample respondent preference in depth with a variety of study formats. ACBC makes this possible for a larger range of questions and experimental designs.
Chris Chapman, Ph.D.
Microsoft Hardware User Research