Results of 2016 Sawtooth Software User Survey

Each year we field a customer feedback survey. This tool allows us to track our achievement in customer service and also track the relative use of different Sawtooth Software conjoint methods.

We fielded the study during March 2016. Among those users whose firms do conjoint analysis, we asked how many conjoint analysis studies they had conducted over the previous 12 months and what percent of the projects used different conjoint analysis approaches. If we weight the results by number of projects completed and normalize them to sum to 100% across the Sawtooth Software conjoint methods, we see the results for 2016 as shown at the far-right within the figure below.

Relative Use of Different Sawtooth Software Conjoint Methods

We've conducted this usage survey since 2003, so previous years are also displayed. Of note, the choice methods (CBC, ACBC, MBC) most recently accounted for 96% of the projects completed, whereas ratings-based methods (ACA and CVA) accounted for 4%. CBC continues to be the most commonly used approach, accounting for 81% of projects.

Project Methods

Percent of projects using different methods: (Weighted data, by number of projects conducted)

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
CBC (Choice-Based Conjoint) 35 46 54 56 59 62 68 65 68 60 65 69 69 72
ACBC (Adaptive Choice) 6 9 10 10 11 11 11
MBC (Menu-Based Choice) 2 2 2 2 2
ACA (Adaptive Conjoint Analysis) 25 20 15 12 11 10 6 6 5 4 4 2 3 2
CVA (traditional conjoint analysis) 10 9 8 7 6 4 4 2 3 4 2 3 1 2
Proprietary version of conjoint analysis 12 10 10 11 12 9 9 11 6 6 9 3 7 6
Self-explicated approach 5 4 3 4 3 3 2 2 3 2 3 3 2 1
Other software system/approach not listed 10 6 3 7 6 10 9 6 5 11 4 5 3 3
BPTO (Brand Price TradeOff) 3 5 3 3 3 2 2 2 2 1 2 3 1 1

*Among respondents who reported that their company conducts tradeoff/conjoint/choice or some other preference modeling. The figures for 2005 sum to only 96% because we included MaxDiff as an option and it captured the remaining 4%. In 2006 and later, recognizing the MaxDiff is often used in addition to conjoint analysis within the same study, we asked the usage question about MaxDiff as a separate question.

Types of Conjoint

Percent of total users reporting that their firms used each conjoint-related software system within last 12 months:

2013 2014 2015 2016
CBC 66% 68% 67% 66%
ACBC 29% 32% 28% 28%
ACA 13% 9% 9% 8%
CVA 6% 6% 4% 5%
MBC 10% 10% 10% 11%
SMRT 46% 42% 41% 39%
Online Simulator 9% 14% 14% 17%
Advanced Simulation Module (ASM) 13% 12% 13% 9%

CBC Project Types

Among those that used CBC in last year, what percent of CBC projects utilized:

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
Constrained estimation 20% 19% 22% 19% 29% 29% 23% 24% 29% 26% 25%
Alternative-Specific Designs * * 22% 18% 26% 31% 26% 23% 27% 26% 28%
Dual-Response None * 6% 18% 15% 18% 22% 20% 23% 22% 25% 31%
Free Format Layout * * 12% 10% 23% 24% 15% 23% 20% 23% 21%
Interaction Effects 26% 24% 26% 18% 20% 24% 19% 17% 17% 18% 17%
Shelf Display * * 6% 5% 9% 16% 9% 11% 9% 11% 9%
Constant Sum * 7% 12% 7% 11% 14% 9% 10% 10% 10% 9%
Best-Worst CBC * * * * * * * 9% 6% 13% 10%
*Either not available in the software or not asked about in that year's survey

MaxDiff (Maximum Difference Scaling)

The table below shows the percent of our total customer base (which involves many non-conjoint users) who reported that their firm had used MaxDiff during the previous 12 months.

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
8% 18% 24% 31% 37% 47% 52% 54% 57% 67% 68% 70%

Sample/Online Providers

We also asked our users which Online Web Sample/Online Panels they primarily used over the last 12 months. Up to six mentions were allowed, asked in open-end format with no pre-coded responses. Sample providers who were mentioned by at least 3% of the sample in 2016 are reported below. (If a firm acquires another firm, the results for years prior to the acquisition are not netted.)

  2008 2009 2010 2011 2012 2013 2014 2015 2016
SSI/uSamp/Instantly/Opinionology 21 22 23 27 28 28 31 38 49
e-Rewards/Research Now 23 30 46 47 47 48 50 46 45
Lightspeed Research/TNS/GMI 6 9 8 10 8 17 17 18 17
Toluna/Greenfield/Ciao 17 23 23 17 15 13 19 18 10
Qualtrics 2 5 3 6
Bilendi/M3 0 1 1 0 2 4 4
Cint 1 2 5 4 3 3 4
EMI Research Solutions 0 2 2 3 2 3 2 4

Lighthouse Studio

Lighthouse Studio (formerly SSI Web) is our flagship software for producing and analyzing online and offline surveys. It contains modules for general interviewing, choice-based conjoint, adaptive choice-based conjoint, adaptive choice analysis, choice-value analysis, and maxdiff exercises.

Try Lighthouse Studio

  Buy Lighthouse Studio