Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to detect outliers in CBC?


I am analyzing the data of my CBC and I want to detect some outliers.
Is there something like a threshold for "speeders"?
The median of elapsed time is around 8 minutes. But I have some respondents, who only needed 2-3 minutes. I also have some respondents, who needed over an hour. But I find it really difficult to say from that, who filled in the survey seriously and who didn't. I mean it could be, that someone was interrupted during the survey and finished it later.
But if the median of the elapsed time was 8, I guess 2-3 minutes are not realistic right?

Do you have any advice or is there a Sawtooth paper about rules if thumb I can use?

Thank you in advance.

Best regards,
asked Aug 23, 2019 by Mary (340 points)

1 Answer

+1 vote
Hi, Mary.  I'm aware of a general research rule of thumb I've seen folks use in the past, which is that times less than 40% of the median may be considered speeders.  

This measure is not particularly correlated with other measures of data quality.  When I look at CBC respondents who straightline (and who I therefore know are cheaters) or MaxDiff respondents with very low fit statistics (again suggesting random behavior) these people are not all and only the speeders based on a comparison with median time.

I'll often use two or more criteria to identify bad respondents.
answered Aug 23, 2019 by Keith Chrzan Platinum Sawtooth Software, Inc. (90,475 points)
Thank you very much, that helped a lot.