I have conducted a CBC and analyzed the results using Logit and LCA. I have already determined the significance of several attributes (using the -2 Log-Likelihood test) and of all my levels (using t-ratios).

Now I was also told that I could check the validity of the model using fixed tasks. Therefore, I have included 2 fixed (holdout) tasks to see if respondents filled in the CBC 'seriously'. Those tasks have the exact same three concepts (and a none option), with identical levels.

My question is; Is it okay for me to just count (in SPSS or Excel) how many respondents chose the same concept in both fixed tasks and divide this amount by the total amount of respondents to get to a percentage and call this a 'hit' rate? Or does some sort of test exist that I can use to determine the validity of the model in a %, using the fixed tasks?

Also, if the simple counting method is correct, what is a desired % of respondents that chose the same concept in both fixed tasks?

Thank you in advance!

Floor