Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Huge standard errors in latent class analysis using low efficient design

As a pilot, we created a small 3^6 design with 6 choice sets. All attribute levels had a clear ordinal rank order. Due to several reasons we just used one questionnaire version for paper & pencil-interviews. Each level appeared 6 times in the design and 2 times with each other attribute level. We were aware of the warning in the Sawtooth Test Design Report that indicated low design efficiency. But we just wanted to get an idea of the atribute/level importance and the expected standard errors with the simulated data were low. The response rate, however, was great and unexpectedly high. Then we used Aggregate Logit within Sawtooth Analysis Manager and we got what we were looking for.  However, with almost 200 respondents we wanted to take a deeper look on the data. We think with Aggregate logit we reached the limit with our small design and as expected with latent class analysis we ran into very huge standard errors. (Using linear coded attribute levels works just fine.)

Even when we didn't want to go there, is there any possibility to get more reliable standard errors in latent class analysis with our design?

Thank you very much!
Andrew
asked Jun 1, 2018 by Andrew Bronze (1,045 points)

1 Answer

0 votes
 
Best answer
It sounds like you already know the source of some of your pain.  But, I want to make you aware of another issue.  If you have relatively small sample sizes and you try to run a high-dimension solution with many classes in Latent Class, the algorithm is able to find small groups that are nearly in perfect agreement (very little error) in what is picked and what is rejected.  When choice probability for a level or levels becomes nearly 0 or nearly 100% within a class, the logit estimates begin to grow uniformly large and begin to separate dramatically from zero.  As these logit-scaled parameters expand, the standard errors expand commensurately too (to preserve the same view on your signal to noise ratio).
answered Jun 1, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (160,485 points)
selected Jun 1, 2018 by Andrew
Brian, thank you very much. And indeed, beyond the design problem, we have this issue with a very small latent class in the 2-class model here. We didn't try a 3-class model as sample size is already small. Also, in former latent class analysis we were confronted with the issue you described which made it difficult to say if the probable high agreement in a small class occured due to strong homogeneous preferences among respondents or any bias.
This was again very helpful.
...