Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

HB subgroup analysis ok if one group much larger?


I am wondering if it is acceptable to complete an HB subgroup analysis if one group is substantially larger than the other (28 respondents vs. 7).

Also - is it important to test the significance of the difference in average part-worth utility values between the groups? Is there a relatively easy way to do this within Lighthouse Studio or another program?

Thanks so much!
asked Jan 10 by chelseav (190 points)

1 Answer

0 votes
Typically I do not suggest breaking the sample into groups that are analyzed separately via HB unless the sample size per group is at least 400.  A paper many years back examined the issue of breaking HB into subgroup analysis vs. running HB on the entire sample, and the results (given the sample sizes employed over multiple studies) showed no improvement (in hit rate) for HB by subgroup vs. HB overall...followed by post hoc dividing into groups.

However, if the main goal is to explore the differences between groups and to more appropriately test for differences between subgroups, then the covariates approach for HB is the more technically supported way to go about this.  It involves specifying the grouping variable as a covariate and then examining the resulting alpha file (the draws from the upper-level model) after convergence.

However, this is not typically done unless there are at least 50 to 100 respondents per segment.  28 respondents vs. 7 respondents seems awfully thin.
answered Jan 10 by Bryan Orme Platinum Sawtooth Software, Inc. (159,810 points)
Thank you so much for your feedback. I am a graduate student doing a small study with local patients with a certain type of rare genetic disease, so the sample size is certainly much, much smaller than would be optimal! There is no previous work with patients in this group, so I thought this would fall under the category of exploratory/investigational, and could generate hypotheses for future work. Because of this, I thought a sample size between 30-60 respondents would be acceptable based on my understanding of your work "Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research."

Do you have any guidance, or references I can refer to, on the types of analyses that are acceptable to complete for studies in this exploratory/small sample size category? Or, would I simply frame any results in a particular way - given all of the limitations of a small sample size?

Thanks so much.