Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Design Efficiency When Combining Prohibition and Alternative-Specific Attributes

Hi everyone,

within my current CBC study I found that for presenting realistic choice sets and still following the goal of the study, it might be necessary to use a combination of prohibitions and alternative-specific attributes.

Going through the Lighthouse help I found that in general such combinations should be avoided as they can lead to unexpected results.

Now I'm looking for some advice if the current setup of the study will yield valid results.
Can this be analyzed by looking at the design report? (see below)

Prohibition: Att1/Lev1 not in combination with Att2/Lev2,3
Alternative-Specific: Att5 only in combination with Att2/Lev2

Thank you very much in advance for any hints/explanations.

Kind regards,
Philipp


Build includes 120 respondents.

Total number of choices in each response category:
Category   Number  Percent
-----------------------------------------------------
       1     412   34.33%
       2     404   33.67%
       3     384   32.00%

There are 1200 expanded tasks in total, or an average of 10.0 tasks per respondent.


Iter    1  Log-likelihood = -1313.14471  Chi Sq = 10.38007  RLH = 0.33478
Iter    2  Log-likelihood = -1312.87947  Chi Sq = 10.91055  RLH = 0.33485
Iter    3  Log-likelihood = -1312.86773  Chi Sq = 10.93404  RLH = 0.33486
Iter    4  Log-likelihood = -1312.86725  Chi Sq = 10.93500  RLH = 0.33486
Iter    5  Log-likelihood = -1312.86723  Chi Sq = 10.93504  RLH = 0.33486
Iter    6  Log-likelihood = -1312.86723  Chi Sq = 10.93504  RLH = 0.33486
*Converged


          Std Err    Attribute Level
  1       0.10658    1 1 Immer Grün
  2       0.06425    1 2 Stadtwerke Musterstadt
  3       0.06318    1 3 Pink Strom

  4       0.09970    2 1 Standard Strom ohne Label
  5       0.06036    2 2 Öko Basis
  6       0.06242    2 3 Öko Nachhaltig

  7       0.05302    3 1 0 Prozent
  8       0.05302    3 2 5 Prozent
  9       0.05286    3 3 10 Prozent
 10       0.05375    3 4 15 Prozent

 11       0.04123    4 1 0 Prozent
 12       0.04043    4 2 50 Prozent
 13       0.04112    4 3 100 Prozent

 14       0.08617    5 1 Grüner Strom
 15       0.08674    5 2 TÜV Süd
 16       0.08562    5 3 OK Power
asked Nov 26 by Philipp

1 Answer

0 votes
The advice int he help menu is that you should be careful to test your design when you combine prohibitions with alternative specific effects, to make sure you don't have big problems.  

Looking at the design report, it looks like all of your effects are estimable (you have standard errors and not *******s, and your model converged without a warning about a deficient design or a ridge adjustment).  

You might want to look at the D-efficiency (the "strength of design" for your model compared to a model with no prohibitions and no alternative-specific effects just to see the reduction in your effective sample size, but given this design report I'd be comfortable fielding this study.
answered Nov 26 by Keith Chrzan Platinum Sawtooth Software, Inc. (78,825 points)
Thank you very much for the quick reply Keith!

In addition, I'd like to ask something about the D-efficiency that you mentioned. As I understood, those values can not be compared between different designs.

How about two designs that only differ in the number of random choice tasks? Is a direct comparison of the D-efficiencies between those two designs allowed?

Thank you very much!
Actually, D-efficiency CAN be compared across designs for the same number of attributes and levels.  In fact, the easiest way to interpret it is as a relative measure.  For example if one design has a D-efficiency of 400 and another has a D-efficiency of 300, we can say that the former is 33% more efficient than the latter.  

You can compare designs with different numbers of choice tasks.  

What you cannot do is compare designs with different sets of parameters (attributes, levels, interactions).
...