Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

effects coding - which attribute level is the reference?


Variables are effects coded in Lighthouse studio when it comes to Logit analysis.

Assuming this, I have two questions:
1) which level is taken as the reference level? E.g. there is an attribute with 7 levels (1,2,3,4,5,6,7). Does Sawtooth Lightouse Studio take the first (1), or last (7), or another one? (and which and why).

2) If the attributes are already effect coded due to the development process of the design file with NGENE and e.g. Level 1 (of 7) was defined as reference level, does Lighthouse studio consider this, or is this effects coding not relevant any more for the further process of analyzing in Sawtooth?

Thanks a lot for your help. I searched the forum (and the help) but didn't find a satisfactory answer.

kind regards,
asked Dec 4, 2018 by bs77 Bronze (730 points)

1 Answer

0 votes
In our CBC programs, the last level of each attribute is specified in the design matrix as the omitted level.  This is only in the design matrix, which is behind the scenes and not observed by our users.  Then, when we write out the utility parameters, we expand out the vector of utilities so that all levels have a utility attached to them (and the sum of utilities within each attribute is zero). So, it's all hidden to the user what was done behind-the-scenes in the design matrix.

In most cases (standard level-balanced designs), it doesn't matter which level one uses as the reference level.  You'll get essentially the same result (at least there shouldn't be statistically significant differences among runs coded different ways for the reference level, as long as you run out far enough to convergence).

If you use a program like NGENE to design your CBC experiment and then import that design into Sawtooth's programs (Lighthouse Studio CBC) for fielding the study, then you will be giving the design as level indices for import to our software.  For example, if there are 7 levels of the first attribute, the design matrix will just be listing numbers 1-7 in a single column for that attribute (in the .CSV file).  Only later during utility estimation will our programs deal with expanding that into an X matrix with 7-1=6 columns.

If you are skipping our data collection process and just moving designs and respondent answers into our HB or Latent Class standalone programs for utility analysis, then again you'd typically be using a .CSV file where you specify a single column of values 1-7 to represent a 7-level attribute, and you'd let our software do the effects-coding and our software would choose which level to use as the reference level.  The manuals for CBC/HB and Latent Class standalone give you the layout of the .CSV file for specifying the design matrix per respondent and respondent choices.

However, power users can entirely control the coding of the X-matrix if they want to, using "user-specified" coding, where the power user puts the 1, 0, and -1 codes in the K-1 column coding per attribute.  In that case, you have control of which level is the reference level for each attribute for CBC/HB and Latent Class estimation.  And, when the utility file is written out by our programs, then you would have K-1 parameters per attribute rather than K parameters per attribute as normally would occur if you let our software handle the effects-coding procedure.
answered Dec 4, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (163,515 points)
ok, so if I understood this right, and please correct me if I didn't:
- Lighthouse studio does not need to consider the NGENE settings in the analysis procedure because the import of the design file is basically just a .csv (and not the formula that led to the design in NGENE).
- It is not so interesting for a non expert user to know which level is taken because it doesn't make big difference.

Is there a difference if the design is based on a multinominal logit  model concept (mnl d-efficiency)?
Lighthouse studio cannot consider the NGENE settings for which level was the reference level.  Not interesting for the non-expert user to know which level is being considered the reference level.  I don't think it would matter for our software to know which level was considered the reference level in NGENE during the design process when the efficiency search was based on d-efficiency.
ok thanks for your help!