Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to assess model quality of utility values of CBC

Hello Sawtooth-Team,

I make a CBC to derive willingness-to-pay later from the part worth values. Now it would be useful for my research to know how significant the part utility values are. I read that you can do tests for it and get the pseudo-R2.

Which tests can be done for this and are they integrated into Sawtooth or do I have to download the data and run the tests with another software like SPSS?

Are there any literature and/or instructions?

Thank you in advance!
asked Feb 13, 2019 by Katrin

1 Answer

0 votes
Check this article out, as it provides lots of details for all sorts of significance testing:

answered Feb 13, 2019 by Bryan Orme Platinum Sawtooth Software, Inc. (171,365 points)
Hi Bryan,

thank you for your answer. I've probably expressed myself ambiguously:
In Discover I get utility values and standard deviations. This is what I need to derive the willingness to pay later. This data would also be sufficient for the independence test (the t-test). What I lack is a measure of model quality for the utility values, i.e. the p-Value like in the link you sent. This is not displayed in Discover.
That's why I wanted to know if it's integrated in Sawtooth or if I just can't find it because it's somewhere else.

But if they're not integrated, I'd like to know if I could do a HB estimation myself with separate software, and if so, which test exactly should I take?

Sorry for the misunderstanding and best regards,
Discover CBC is our streamlined and easier-to-use version of CBC (as compared to our Lighthouse Studio version of CBC).  Discover CBC has quick Empirical Bayes estimation, which produces results quite similar to HB in our testing.  But, HB definitely gives you more horsepower and better results for serious work in estimation and statistical testing.  Discover (in its advanced export area) gives you the ability to export your CBC data in a file format appropriate to read into our CBC/HB standalone estimation software.  From that, you could get the draws data (rather than the collapsed point estimates) needed to perform the most sophisticated types of HB testing described in that Chapter 12.  

Again, I'm assuming you are really wanting to get more sophisticated and fully use HB estimation and testing, rather than using Empirical Bayes and the standard Frequentist type statistical testing.