Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Questions regarding t-test and IIA assumption in latent class


I am currently completing my master thesis and I struggle with two things.

First, I have performed a t-test in which I compared the utilities of different levels of one attribute, so I could conclude whether one level is preferred to another. Is this really necessary, or can I conclude from the utilities and the given t-ratios whether one is preferred over another?

Second, in my thesis I state that the multinominal logit model suffers from IIA, and I therefore supplement this analysis with a latent class analysis and HB-model. Preparing for my defense I would like to know why the latent class and HB suffer less from IIA.

Many thanks in advance!

asked Oct 20, 2017 by Anoek (120 points)

1 Answer

0 votes
For an intuitive explanation of IIA and how latent class reduces IIA problems, while HB reduces IIA problems even more, please read the following white paper:


Regarding the t-test for differences of levels within attributes, I recommend you do a Bayesian statistical test using HB.  It's actually quite easy to do.  Here's how you do it:

Run HB.  Make sure to request the additional advanced file called the "alpha draws" (look in the advanced settings of the software for this output option).  The alphas are the estimates at each iteration of the algorithm of the population mean utilities.  For example, you might run 20K total HB iterations.  The first 10K are ignored as being too early in the procedure to use (prior to convergence).  So, pay attention only to the last 10K iterations (pay attention only to the last 10K of the draws of population mean utilities in the last 10K rows of the alpha file output).  The alpha file where these draws are stored is a simple to use .CSV file, where the alphas are listed one vector of utilities per row (one vector per iteration).

Now, simply count for what percent of the alpha draws (after convergence is assumed, so from 10001st through 20000th draws of alpha) one level is higher than another level within the same attribute.  For example, if you found that 97.6% of the alpha draws after convergence show that level 1's utility is higher than level 2, then you are 97.6% confident that level 1 is preferred to level 2 by the sample.

If you must perform this test only within the respondents who have the highest likelihood of belonging to a class (from latent class analysis), the simplest (but also a good) approach would be to use the latent class membership as a filtering variable and only run HB to include respondents within that class.  Repeat the same counting procedure across draws of alpha as I suggested to perform the statistical test.

There is a more advanced way to do this using your latent class membership as covariates in an HB run, but this complicates matters a bit more as you interpret the results of the alpha file to account for different respondent groups.  It's not terrible to do, but just more complicated than what you may need to do for the scope of your work.
answered Oct 20, 2017 by Bryan Orme Platinum Sawtooth Software, Inc. (174,415 points)
Oh, that 1998 paper I reference refers to "ICE", which was an earlier shortcut method to obtain individual-level utilities.  Now, we use HB.  The same argument we make for why ICE reduces IIA problems relative to aggregate logit or latent class holds for HB.