Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

When is it worth it to include a interaction effect in ACBC?


When analyzing the results of my ACBC study, there was one interaction effect identified. This interaction proved to (only just) significantly improve the model fit if it was to be included in the analysis. However, the amount of parameters to be estimated  in the HB analysis would increase from 28, to 48. Is it worth it to include the interaction effect anyway, despite the massive increase in complexity of the model? I know the complexity of the model is a big factor in CBC, but how does this work for ACBC? Since the amount of parameters are often much larger by nature, due to more included attributes.
I'd also like to add that i'm using the ZC individual part-worth utilities to create segments, using Latent Class Clustering. Including the additional levels of the interaction levels might be troublesome.

Additionally, when analyzing the HB results (interaction included), i noticed something strange. When examining the part-worth utilities of the interaction levels, is saw one combination of attributes which indicated as a prohibition with a large and positive utility value. I'd really like to know how this is possible, and what could have caused this? Thank you in advance!
asked Sep 6, 2019 by ash (180 points)

1 Answer

0 votes
If you were using aggregate logit to analyze the potential benefit of an interaction effect, then later if using HB in most cases we see such interaction effects go away.  In other words, the interaction effects observed in aggregate analysis are usually due to unrecognized heterogeneity.  Once we recognize heterogeneity via the HB model, then the interactions tend to not be useful anymore.

However, if you used the "Model Explorer" (a standalone tool that communicates with the standalone CBC/HB tool) to find that certain interaction effects indeed provide a reasonably large lift to hit rate with HB, then that's a different matter.

Also, if you have a prohibition between two attributes, then it wouldn't be proper to ask the software to estimate an interaction effect between those attributes.  It will try to estimate parameters for attribute level combinations that were never shown to respondents.
answered Sep 6, 2019 by Bryan Orme Platinum Sawtooth Software, Inc. (174,415 points)