Dear ACA/Web v6 user,A bug (Bug #1 below) has been discovered in ACA/Web v6 that affects you if you have been using the advanced options in ACA for customizing or dropping the Importance questions. Two other bugs (Bugs #2 and #3 below) are more obscure and may have affected you if you were also dropping Importance questions. We sincerely regret these errors. Please read all three sections to determine if these bugs affected you.
Bug #1Affects you only if you...
1. Are dropping the standard importances questions in ACA/Web v6 by clicking (from the Specify ACA Settings Dialog) Skip Importances Questions (Advanced) + Use Group Means for Prior Importances (or) Set Prior Importances Equal for All Attributes (or) Set Prior Importances Based on Other Questions.When you drop the importances questions and use group means, equal importances, or customized importance questions based on other questions, the design algorithm for choosing the pairs did not operate exactly as intended. Furthermore, if you were dropping the importance questions and used any of the functions (such as [%ACAImportance(attribute#)%]) within questionnaires to display the utilities or importances in real time to the respondents, these were computed incorrectly. (But the final computed utilities under OLS or HB as used in the simulator were not affected by this bug and were correct.)
This bug caused the routine for updating the utility scores (after each page of pairs questions are submitted) to use improper scaling for the pairs questions relative to the scaling of the prior importances. Despite the bug, the order of the relative importances for the attributes were preserved, as well as the relative utilities of levels within the attributes, but the updated importances were too flat. This did not affect the design for pairs questions showing just 2 attributes at a time. However, pairs questions showing 3 attributes or more at a time would potentially have had less utility balance than intended within ACA. We sincerely regret this error. The good news is that the unintended pairs designs still would have featured excellent level balance and valuable tradeoffs (the pairs were not dominated by having all good or bad features on one side). In addition, our own research that we presented at the 2004 Conference (where we showed benefits for dropping the importance questions versus standard ACA) were also affected by this bug. Those designs worked well. Any negative effects due to the problem in the designs were probably minimal.
Affects you only if you...
1. Are using a computer to estimate utilities under OLS or HB where the region is set to one which interprets the decimal place as a comma (,) rather than a period (.). For example, German computers do this.
2. Use the Custom Importance Questions to substitute for the standard ACA/Web Importance Questions (ACA Settings + Priors + Set Prior Importances Based on other Questions).
When those two elements combine, the resulting utilities can be incorrect. To understand the issue, consider an Importance Question customization where (for example) a 15-point scale is used rather than the default 7-point scale. If a respondent answers a 7 on a 15-point scale, this is converted to the standard 7-point scale in the data file (dat.dat file) via the transformation 7/15 * 7 = 3.26666666666667. If you downloaded the data to a Windows machine set to interpret commas as decimal places, it would interpret that number as 326,666,666,666,667. However, a value of 12 on the 15 point scale would be converted to 12/15 * 7 = 5.6, which would be interpreted by that same Windows machine as 56. So, you can see how the importances feeding into either OLS or HB estimation (inside SSI Web or via .acd export) could be misinformed.
Affects you only if you...
1. Use customized importance questions to substitute for the standard importance questions.
2. Your customized importance questions provided data that included decimal values.
ACA/Web's code was written assuming that your input for customized importances would be integers without any decimal places (prior to the conversion to the 7-point scale as spoken of in Bug #2). If a decimal place is encountered, ACA/Web converts the entire value to zero. If all customized importance values are zero, ACA/web saves these to the data file as 3.5 (half-way on the default 7-point scale). The error would have affected both the utilities used to design the pairs questions and also final computed utilities via OLS or HB.
We sincerely apologize for these bugs. You can download a free update to SSI Web v6.4.2 on our website at www.sawtoothsoftware.com/downloads.php. If you are currently fielding a study with ACA and want to update your study while it is fielding, please give us a call.
Please call us if you have any questions at 360/681-2300.