Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

timing analysis in Conjoint and Maxdiff

I have been discussing some new approaches with the team, mainly involving timings into our analysis and have a couple of questions that I hope you can clarify for me:

1.    Do Sawtooth know of clients who’ve used response time in their analysis?  

2.    Is it possible to collect the response time for each selection in Sawtooth?  For Maxdiff/Conjoint?  No bugs or other unexpected problems?

3.    You mentioned to me before that it is possible to get a timer for the Conjoint/Maxdiff but not force to progress to the next page without an answer, would you have a javascript example for this timer? And how would you implement this?

4.    If I understand it correctly at the time of analysis Sawtooth takes the experimental design and the binary response data then passes these into an HB module.  Is there anything either practical (in terms of the data complexity) or statistical (in terms of providing non-binary data) that would prevent us from taking the binary response data, adjusting for the response time (e.g. fast response times result in data close to 0 or 1, slow response times result in something closer to 0.5) and then feeding this into the HB module?  And are there examples of this?

5.    We are currently using version 6.6 for conjoint analysis and I am very keen to upgrade to the latest version, would you be able to highlight the main differences that would help me in my business case for my boss?

Thanks very much.
Kind regards,
asked Apr 15, 2015 by anonymous

1 Answer

0 votes
Best answer
Hi Caroline,

I'll go ahead and answer your questions in order.

1. We aren't really aware of any of our customers who do this.  From some of the research we are aware of, people have tried to incorporate response time and the results aren't very compelling.  For example, it's difficult to know if a fast response time was because the choice was so obvious to the respondent, or they were speeding through the survey.  Likewise, a long response time might indicate deep thought, but the task is very difficult with multiple "good" options, and in the end it's a coin toss for the respondent which option they go with (or maybe the cat jumped on the keyboard and minimized the browser).

2/3.  Current versions of the software do automatically record page times for respondents, so you're in luck if you want to pursue this further.  To clarify, though, page times are when the next button is clicked, not when the radio button of a question is selected (which might be considered a better measure of when the question is completed).  It would be possible in theory to build something like this, but would require you to use Free Format questions to customize CBC and MaxDiff questions so that you could add some JavaScript to push values into variables behind the scenes when something is selected.  You'd also have to consider how to handle changed answered if you went with a customized approach, which sounds messy!  I don't think we have any examples of this, but one of our developers could probably do this as a small, paid customization if you really wanted it.

4. Technically we record the number of the response in the data (profile 3 was selected in task 1, profile 4 was selected in task 2, etc.) and push forward the design and that answer to the analysis routines where it's all handled automatically.  The current, standalone versions of tools like CBC/HB can take in CSV files of binary data, but if you give it something like a 0.5 as an answer, it's going to flip into chip allocation mode and normalize the answers so that they sum to 100.  This means your .5 goes back to a 1 since you didn't really have chip allocation data.

5.  We're currently on version 8, so there's a lot that has changed so far (version 6 was originally released in 2006).  Some of the changes, like MaxDiff designer improvements, have short papers written about them going into a lot more detail so don't hesitate to get in touch.  The biggest improvements are:

-No more 4 character study names
-Multiple conjoint and MaxDiff exercises in the same survey
-Formal back button that reloads answers even if the respondent leaves and comes back
-Graphical select/navigation buttons and overall style improvements
-Randomized Blocks of questions
-Translation support via XLF documents (translation document standard)
-Auto-Advance questions where constructed list logic results in just a single option to choose
-MaxDiff received on the fly individual estimation and paper and pencil import support

-Mobile enhancements (SSI Web detects small screen sizes and adapts the presentation of questions for mobile devices automatically)
-Utility estimation for CBC is now inside SSI Web
-Adaptive CBC received an overhaul with a better designer and support for some advanced tricks like alternative-specific designs and merging rows
-CBC conditional displays now allow for more than 4 attributes and missing attributes
-Best/Worst CBC response type added
-Automated question counters for CBC and MaxDiff
-MaxDiff has a new design algorithm that produces better balance for each respondent
-Added support for Anchored MaxDiff data
-You can copy and paste questions between multiple instances of SSI Web
-New constructed list instructions like AddSorted and Mirror were added
-A fully functional data generator was added to test your surveys automatically
-Question looping added
-Semantic differential question type added
-Automated survey upload to our hosting system
-Survey hosting is now free with annual subscription licenses

If you have any questions on anything above, upgrade costs, etc. give us a call or shoot an e-mail in to support at sawtoothsoftware.com
answered Apr 15, 2015 by Brian McEwan Gold Sawtooth Software, Inc. (41,570 points)