From Data to Decisions: How to Present Conjoint Results That Inspire Action

Automatically transcribed

Vanessa: Quick disclaimer before we start, this episode was originally recorded as a webinar and edited for podcast format. You can find the original webinar recording on our website at sawtooth. com or on our YouTube channel.

Now back to the show.

Brian McEwan: All right. We're happy to welcome you to today's webinar from Data to Decisions, how to present conjoint Results that Inspire action and Build Trust. Now I am delighted to introduce our speaker today, Megan PIs. She's the founder and CEO of Num Marius. She received her master's in mathematics and statistics and has been doing this stuff for a little while. A lot of years doing conjoint, MaxDiff segmentation, new product research.

She's been published in the Journal of Choice Modeling. And if you've ever come to the Sawtooth Research Conference, you'll recognize her. 'cause she has a history of winning best paper awards for some of the new and interesting boundary pushing research that he, that, uh, Meghan does in this area.

So we are thrilled to learn from stories from the trenches and all the tips and tricks and things that she is learned over the year, uh, over the years. And so, Megan, we will turn the time over to you.

Megan Peitz: Thanks, Brian. I glad you said over a little years and didn't, uh, stress my age.

So Brian gave me a great introduction so I won't spend too much time here, but, uh, my company is called Numer. It's a mashup of the words numerically curious. And if you're an Uber nerd like myself, you may notice that in some of the numbers in our backdrop. Is a pattern. Each of these four lines represent a very important mathematical number.

That is how nerdy I am. Uh, but why am I here today? Well, at Numer we get to run hundreds of conjoins a year for some of the world's biggest brands and the most innovative startups. And these are some of those brands. And so you can imagine that after six years of being in the room virtually in the room with product planners, product managers, and C-suite executives, I've learned some key lessons in what works and what doesn't when it comes to presenting conjoint results.

And that's why I'm here today because I love this methodology so much, and I want every single person who runs a conjoint or commissions a conjoint to realize the value in the data. Because as a research vendor myself, I have been in your shoes, right? So many of you on this webinar are probably responsible for the actual analytics involved in a conjoin.

So I might be speaking directly to you when I talk about a time when you built a beautiful design, spent hours cleaning the data, thoroughly, getting a model that's extremely predictive of your fixed tasks, and creating a beautiful simulator tool that calculates share and revenue and has all the checks, boxes, and the filters.

But after presenting the results in the report, you see that blank stare on people's faces in the room, and you can tell that they don't really know what to do next. Or maybe you're an insights lead at a big brand and you're frustrated because you've spent over six figures on a conjoint project in the past, hoping for that clear guidance on how to prioritize your product roadmap or choose a price point only to feel a little lost in the deck that shows maybe utility scores and is full of charts and stats, but no clear direction.

So you're left holding this bill for a really expensive piece of research and nothing that you feel super proud to take to your stakeholder. Even worse, you may avoid running a conjoint in the future because of this experience. So I can't see the chat. I, uh, but if you, if this is resonating, feel free to put a plus one in there, or if you've been in this situation before because at the end of the day, like the, I don't mean to sound doom and gloom, right, but the hard truth is that the best data means nothing.

If it doesn't inspire action. So I'll repeat that again. The best data means nothing if it doesn't inspire action. And in my opinion, most contract results aren't failing from bad models, but they're failing from misalignment and bad storytelling. And what's at stake with that? You're gonna lose the trust of your stakeholders.

You're gonna lose the momentum you had for that product or service, and you're gonna have dust collecting on your decks and, and no repeat customers,

right? But we are gonna fix this today. I'm gonna walk you through the framework that we use at MIUs to go from data to decision making. And if you joined us a bit late, uh, you can actually capture the QR code on the next screen and you'll be able to see, uh, the entire deck and the framework.

So let's unveil the, the framework. Uh, our framework has five main pillars to making a great conjoint deck. Align, translate, simulate, visualize, and activate. And I'm gonna give you all the tips and tricks through each of these phases. And if you can follow this framework on your next project, I guarantee it will be so easy to deliver a presentation that's gonna drive confident decisions.

So let's start with the first pillar. First up is alignment. Now I know we're talking about presenting conjoint results, and yes, this piece of the framework is in reference to in your slide deck. But you'll be more successful if you've spent the time upfront in the project scoping phase, getting alignment on which decision the research is helping make, or what the stakes are if you get that decision wrong.

And what success will look like at the end of the research. A good piece of research will make sure that we have this alignment upfront and continue to remind the client of those main objectives throughout the survey design, throughout the creation of your attributes and levels throughout the entire process, all the way up until sharing the final results so that you know what you're gonna give them and they know what they're gonna get.

And so every good deck at Numer is gonna start with a slide on, we did this research to X, right? Maybe that X is justify raising our prices by $5, or validate that we should launch SQB instead of SQC alongside our current SQA. Okay. So if you're the analyst and you're unsure on the why behind the conjoint, then you need to make sure you get to the heart of the matter because you don't wanna give your boss or your stakeholder something that they can't take action on or doesn't answer that why?

Because if someone just says, we need to understand which features customers care about there are multiple methodologies that could suit that business objective, right? And conjoint might actually not be the best fit, but we're not here to talk about that. We've already decided that that conjoint is the best fit.

So you need to double down on what they mean by which features customers care about, right? What is their why? Is it for feature prioritization? Is it for messaging? Is it for the go to market strategy? Are we trying to optimize the features for share for revenue, or for both of those things, right? Because without that clear decision path, we might end up making recommendations that are irrelevant, right?

Hey, price your product at $99. And then the CEO in the room says $99, we can't even make it for $99. You have to charge at least 1 29, right? And then the rest of the meeting is basically worthless because now they've already tuned out, right? So you need to have a deck with results that people are gonna know what to do with them.

Another big alignment myth that I've seen is stakeholders trying to answer too many objectives in one study. Now I get it, you're paying for people to take your survey, so why not squeeze in as much information out of those survey takers as possible? But if we don't prioritize our objectives, we could end up with so much data that it's hard to tell a story or a conjoint with so many attributes that there's little variability in the preferences and no clear signal on what to do next with your product or service.

Now I'm not harping on conjoints with lots of attributes and levels because as Brian mentioned, we've won best paper a couple of times on running conjoints with lots of attributes and levels. So we feel very comfortable testing large attribute sets if we have a clear path for the decision that needs to be made with the data.

In this case, we'll design something that that can handle that large attribute set. But the difference is I'm gonna ask enough questions to try to uncover if it feels like my client is throwing everything in the kitchen sink in, because they don't have the confidence to push back on their stakeholders, right?

So then I'm gonna arm them with reasons why maybe we wanna condense the list of attributes and levels and really try and help them prioritize the decision that they need to make with this data today. Are we trying to get insight for the 2027 product launch? Or do we really just need to get crystal clear on our pricing strategy for the up upcoming refresh, right?

That can really help you delineate or cut off any of those 2027 features from your conjoint that you're trying to run to inform the 2026 pricing strategy. Okay? So having this kind of strategic conversation not only makes you look good as a partner, but also helps the client better articulate their goals and their strategies, and it gets you that alignment that you're gonna need come reporting time.

So some good prompts for getting alignment early are asking your stakeholders, what decision will this inform? What are the options you're considering today? What would a good result look like to you? Who else needs to agree with this recommendation to get buy-in? And my personal favorite question. Is, what would you want to be able to say with confidence at the end of this project?

Because then that's saying, let me help you, right? Let me help you look good so that you can stand in front of your boss or your execs or your board and say what you need to say with confidence. Yes, we can raise our prices, X, Y, Z, this is Y. Or you know what? We need to go back to the drawing board because our plan isn't gonna compete with competitors A, B, and C, right?

I want them to be able to stand up and be confident in what they're gonna say with these results. And if we can get the stakeholder to articulate that business problem, then we can all be in alignment on what the conjoint needs to answer. So here are some examples of what you wanna try and get out from your stakeholder.

So you want them to say something like, well, we're deciding whether to raise the price from 49 99 to 54.99, and we don't know if we can justify it. Like that shows that they're dealing with a tension around the decision to raise prices. And based on that one little hint, I'm probably gonna create a fixed task in my conjoint with one alternative of their exact product at 49 point 99, and on the other screen at 54 point 99.

So if all else fails, I got a quick frequency run that says how people will feel, right? Obviously I'm gonna use the conjoint to model the data and do lot, lots of better things, but I'm gonna do everything I can to be crystal clear on that decision to raise their price. $5. Okay. Some other examples here, uh, we need to pick three features for the premium tier that drive the most user value.

So they know that their resource load is three features. They probably can't do more than that, right? And they wanna give engineering or development like that clear signal on those three features to add to the roadmap. So our research better deliver which three it should be. Okay, so the more you can get crystal clear, the better report you'll design and, and ultimately the better experiment you'll design as well.

All right, so now that we've hopefully attained alignment on what it is we're trying to solve, the next step is to bridge that gap between the technical rigor of a conjoint and stakeholder comprehension. In my opinion, this is arguably the most important piece of the puzzle, and that's why I think it's imperative to avoid stats, jargon, and speak in the language of the business.

Stakeholders don't care about utilities. They care about which features matter to their users. So speak in plain language and bring your attributes to life through the lens of the user or the buyer. So I don't wanna call anybody out, but it wouldn't be a Megan presentation without, you know, giving a little bit of slack to showing average utility scores.

In this slide here is what I think is a bad example and it's really just a dump of the average utility scores that you get out of a, you know, statistical platform that runs contract models. You get more outta saw two software, but of course this is one of those outputs that just comes kind of standard, right?

And so you're looking at the average utilities for golf balls by three attributes, brand, performance, and price. And as brilliant as our stakeholders are, they don't know what utilities are. And you certainly don't wanna spend precious minutes of your presentation explaining hierarchical bayesian multinomial logistic regression.

I can tell you that kills the room. And not to mention there's so much room for wrong interpretation. When you look at this chart, for example, if someone who is unfamiliar with the conjoint just picked up this one slide, they might think people love Magnum force brand and they hate long shot, but that isn't true, right?

Utilities are typically placed on a zero centered scale, so there's always gonna be positives and there's always gonna be negatives. That doesn't mean they love the positives and hate the negatives, it's just that the higher the value, the more preferred it is. Not to mention these data are interval scaled, so you can't say that a utility score of a 47 is three times as good as the utility score of 15.

Okay, and one more. Gotcha. If I haven't made it clear enough that my personal favorite is not utility dumps is, is that the utilities are rescaled within an attribute. So you also can't compare the 47. For 15 yards farther and say that that's better than being a magnum force or a high flyer product.

And okay, one more. Did I also mention that this has no competitive context? So if you only had this information, you would say, well, shoot, guess we're pricing at 4 99, right? But if every ball on the market was 4 99 and only drove five yards further than average, then you might actually be able to charge 10 99 for a ball that drives 15 yards farther because you'd be the only one on the market who could claim that.

But this chart here offers no prioritization, no impact. And no business relevance if you can't tell. It gives me so much anxiety that my clients will misinterpret the true findings that I don't even put these in our report. They are nowhere to be found. Maybe in the appendix, and I'll show you how to, how to report them if you're going to in a little bit, but don't put 'em in the main deck, okay?

Also, don't put those standard important scores in the main deck because they too can be misleading. They are literally derived from those utility scores that I just hated on and have no competitive context, so it can't answer the stakeholder's Real question on what we should build. Now, in this example, it looks okay, right?

The important scores are generally similar. Price is most important, followed by performance and then brand. So we're probably not missing too much of the story. But what if you had the results that said price was the most important feature, right? 50% of the important score, and your client is the cheapest in the market.

That would be great, right? But then you go to simulate things in the market and you realize your client has the lowest share. And they say, but Megan, we're winning on price. The most important feature. That's pretty conflicting, right? And I wouldn't blame them for not understanding because what's not being captured here is that perhaps price is the most important based on the utilities.

But perhaps that's because we tested a range of prices so ridiculously high that whenever anyone saw that one super high price point, nobody chose it. Leading to a really large negative utility score and arbitrarily making the importance of price looks super huge. But the only important thing about price is that you can't charge astronomical prices, right?

So what we need is a way to transform these outputs into strategic insights, not just these abstract concepts, because product leaders want to be able to say things like, customers are two times more likely to choose a product when it includes feature A dropping, it would cut interest in half. That statement is concrete relevant to the business and stakeholders can act on it, right?

Or something like customers would require a $5 discount to accept a less customizable version of the product. That's financial language with a number that everyone in the room can understand and ties directly to product and pricing strategy. Or the last one, right? For our premium audience, adding feature B delivers more perceived value than a $10 price cut.

This speaks to the trade-offs that consumers are gonna make at shelf and shows how different groups of people value the product. So this is how your product leader thinks. Wouldn't it be ideal to actually give them a slide deck that talks like this? And the best and only way to do this is through simulations.

So this is where the real magic happens. With conjoint data, we simulate the business impact of different paths the organization could go. We do things like what happens if we raise price, or we show what we gain or lose when we do different options. Okay, so let's take a look at what this might look like in practice.

All right. Let's assume that our client was planning to launch Product B, but they didn't know whether they should replace product A. With product B or keep it on the shelf with product A. Okay, so we are now aligned that we need our data to clearly show if product A should be replaced or kept on the shelf.

With our conjoint results. We can very clearly simulate a scenario where product B replaces product A and another scenario where product B is alongside product A on the shelf at Numer, we also love to level set with the current state, right? So you can see we start with what we call the today case and only Product A is available.

Then we show what happens if product B replaces product A or if product B is launched alongside product A. Because there's also a world where I want to be prepared to say. Actually don't launch product B at all. Go back to the drawing board because that's a terrible idea. So even if their concept was, do we keep product A when we launch product B, or do we just launch product BI as a strategic partner wanna be prepared to say, I actually don't think you should launch product B.

Or maybe your plan for product B isn't super great, here's how you could make it better. Okay. Before I go on to the next slide, which will address the how you can make it better, a couple of other things that I really wanna point out about this slide that I'll also talk about in that visualized pillar is that we are translating these results for our clients into business language.

Our headline is very clear, right? Having both products on shelf optimize a share, and then we have some of those deeper call outs on the side, and we point them to how and where they can see these findings. The parentheses A, when Product B replaced product A, it garners about 5% interest share. And then, well, it's kind of hard to see, but there's a little tiny A in that second column.

And then the, the finding around point B and where we see that and the finding around point C. And I also love including whether, or, I'm not saying I'm the best at creating visuals, however, I think it's extremely valuable to have a how to read this chart box because we know that sometimes our slides, or we hope that our slide is gonna get pulled out and brought into a bigger deck that's gonna sit on the CEO's desk, right?

And so we need to make sure that the CEO has the context. So it's so important that anyone reading the slide can immediately grab it and understand the outcome. You can also see our footnote on the bottom left, which is gonna point them to the full product specs, right? Not only is it gonna say what is product B, what is product A, but it's also gonna point them to how all of the other competitor SKUs are defined and their price points.

So they'll have context, right? We're gonna try to link them to any additional information that they're gonna need to feel confident in the claim that we're making in our headline. Okay, so I had mentioned, I wanna be prepared if I, if I wanna say, Hey, actually don't launch product B at all, or if you do, here's how you can make it better.

That is what I do, leveraging sensitivity analysis. Okay. So especially if a client is in the development phase of the product, they still have a chance to make some tweaks and changes, right? They also really want to know which features are can't miss features, right? And so all of your resources need to make sure that that feature gets into the product before launch, right?

And we do this with nice, easy to follow icons, okay? So we're taking product B and we're keeping the levels of product B the same and just changing one level at a time. And then we're plotting what the change in the share of preference would be. This is what we call sensitivity analysis, okay? And we're doing this in that competitive context.

So if we were to go back one slide, we would see, you know, product B gets 4.8% share when it only it is on shelf. And that is that dotted line that's going across. And so sometimes when I voice over this slide, I say to my client. You know, you can blur your, i's all you have to look at is where those biggest swings are from that dotted line.

The biggest swings up are things you could, should consider adding to the product to drive share. And the things that go down are those can't miss features. Okay? And when you work in the tech space, you know that things can come up, hiccups can happen. You can start to build something and all of a sudden you realize that it's gonna take three more developers and another year in order to get that feature functioning like we had talked about.

Okay? And so if you knew that dropping that feature from the roadmap had no impact on whether people bought product B, then you could be like, okay, let's not spend millions of dollars on devs and resources to make that feature. It doesn't really have an impact yet, right? It might in a couple years, but if your feature we're talking about is maybe attribute eight here.

And you're like, yeah, we don't need to do attribute eight, then I would go and say, oh yes, you do. Right? Because if you drop feature eight from level four down to level one, you will nearly cut your share or your interest for product B in half. Okay? So you can see very clearly we're calling out, attribute nine has great potential to have a lift on product B, but if we don't hit, you know, level four of attribute nine or level four of attribute eight, we're gonna have a real problem on our hands in terms of driving people to purchase.

Okay? And then again, I have the how to read chart, and then I have the hyperlink to the full spec so that nobody hopefully takes this out of context. And then of course, one of the other primary business questions is probably when we launch product B, what should we price it? And so by default we're gonna have the price sensitivity, which is the blue line there for product B.

And you can see the planned price at 4.8% is $139. And so we hold all the features constant and we just change the price. And obviously if we drop the price, more people would buy it, but does that really generate more revenue? And then maybe if our client has costing information, we could actually help them find the profit maximizing price point.

And then I can be very clear in my headline to say Product B could withstand a 10% increase to maximize profit. And then maybe I might have a follow up that says, with minimal impact on share, right? Um, because I think that is probably what's gonna be most important to the business. And then of course my how to read and my link to the competitive set.

So if you've done the job of simulating the business impact. Of different scenarios, you should be able to make claims that your stakeholder is gonna immediately be able to act on, right? This is how product leaders think, and so this is how you should be framing up the results of your research, right?

Make that bold claim. Launching this premium tier at 14.99 drives the most revenue while maintaining strong uptake. Okay? Removing feature A would require a $7 discount to maintain the same interest. Replacing product A with B reduces total share keeping both on shelf optimizes portfolio performance.

This is the language that your stakeholders speak and you need to use the data to translate it into that language. Now, hopefully you saw through the simulation examples how important visualizing the results are. And again, I am not saying I'm the best at creating visuals. I welcome any and all suggestions from those in the audience on how to prettify these even more.

But in general, after presenting over hundreds of these slide decks, I found that simple. Well annotated visuals make or break understanding. So maybe your charts are prettier, but if your stakeholder can't consume what's on the slide in five seconds, they've already moved on, right? I want to make it as easy as possible for them to consume the takeaway that I'm trying to make them walk away with, right?

So I lead with the answer, I highlight it right on the chart. I color code wins and losses, or I use those stop signs and those stars, if it takes you a minute. To explain the chart, it's probably going to take the audience much longer to figure it out. Okay, so some high level tips for clear visual storytelling lead with the headline, not the chart.

Please don't put feature sensitivity in the big headline, right? Make the headline a takeaway. I also try to use only one chart per message. So if you need to show multiple things, spread them across slides. If you noticed, I have simulations, sensitivities, price curves, right? One chart per message, and then highlight the point with codes or callouts or asterisks, right?

Whatever would be intuitive. For your audience, for the, you know, claim that you're trying to make. And I love those annotations, right? Then those brief notes. To increase the speed in which your stakeholder can interpret what they're seeing, okay? If you make their job easier at consuming the data, you will have more time to spend on strategic recommendations, right?

And I think everyone in this room wants to be more a part of the strategy and less about handing over data tables, but I don't wanna speak for everybody. Alright, finally, the last pillar is activate, right? I think I've kind of peppered this throughout, but I am a big proponent of making your recommendation boldly.

Oftentimes at Numer we get hired because we are the third party in the room. We are the people that are there to interpret the data, to allow you to make an informed decision that, you know. We don't have buy-in either way. I am not gonna get paid if product B launches or doesn't launch, right? I'm just gonna get paid if you, accept the research and think it was a good job and are able to make a decision from it.

Okay, so I don't have any sort of buy-in, right? So I should feel comfortable guiding the decision, providing options, right? But leading with a clear point of view and always including context for credibility. The more you've dug into the data and the more you feel confident in the result, then you should make, you should have that translate through what you are saying in the activation phase.

Okay, so a couple of pro tips here. I know activate is technically the last of the five building blocks, but if you've noticed, it's not really like an order sequence, right? Because we're kind of weaving everything together. You have to have all five pillars. Firing on all cylinders to have a really successful conjoint deck.

Okay? So act, the activate piece is typically done kinda after we've done all of those pieces, but the minute that I come up with that activation recommendation, I'm gonna put them as close to the title slide as possible in my full deck. Now, I know some people maybe like to save their key findings until the end, but I would challenge you on that, right?

Why? I love telling my client, here's what my, sometimes we don't even make it past slide three in our presentations, and they're an hour long, and it's the title slide. And here's what we did, and here's what we saw. Literally three slides. And I love telling my client on that, you know, TLDR slide, which I'll give you some examples here in a second.

Here's my findings. If you don't trust me, like you can go see the, you know, the remaining 40 pages of the deck to see how I got here. But trust me, and we can spend time talking about how these findings impact the business or the decision you were gonna make and spend time on that strategy, not on the statistics.

Okay, so here's just one example of A-T-L-D-R slide for a client. If you're not using TLDR, which means too long, don't read, please start. Executive summary is like really old school. So not to date myself, but I started my career saying executive summary, and now I only say tl, dr. Too long.

Don't read. If you can't, you don't have time to read anything else in this deck. All you gotta do is read this one slide. One slide, right. You get the gist. Okay, so in this example, I've obviously unbranded it and changed some of the prices and the context, but uh, this example, and the next two that I'm gonna show you are literally stripped from recent presentations.

So on this one it's very minimal and very clear. We try to keep to three themes and three themes only because honestly, that's like all anybody can ever remember anyways. Uh, so let's just read through this one quickly. The first claim, there is a strong appetite for a premium tier, and you, my client, should strongly consider launching one, right?

I'm immediately addressing that we are aligned on the business question of if they should launch a premium tier. Then I say the more value. My client can squeeze into the premium offering. Obviously the more users will be interested, but keeping the price at 14.99 would be ideal from both a revenue and interest standpoint.

So this is how our team recommends optimizing that premium tier, right? Try to squeeze as much stuff as you can while still being profitable so that you can maximize both interest and revenue in the product. Okay. And then the last point, while this data suggests a fully featured premium offering drives the most revenue, you should leverage internal costing information to make sure the optimal price point is set for profitability, as it may take many resources and time to build out a fully speced premium offering.

So our client did not share with us costing information, and so I wanna make it very clear that sure, the data says 1499 is the right price point. But I don't really know if you can make that product that I want you to make for 1499, right? I don't have that information. But you and your finance team, or your pricing team, or your forecasting team, whoever, should probably do a little more stress testing of our recommendation when you finally think about, you know, resources and things involved.

Okay? So that's one TLDR example. Here's another example. I like this one a lot because we have the three main themes at the top that address the client's business problem of should they move to a bundle strategy over an a la carte strategy. And then we have three bullets to support our three main takeaways.

And what I also like about this TLGR is we are hyperlinking. So any place where you see an underline, you can click on that and it takes you directly to the slide with the supporting data. Making the deck interactive and it's like my proof point of, hey, let's go there. Right? So like I mentioned title slide, here's who we talked to, here's what we found.

And if they're like, you know, got it, Meg, there's, you know, opportunity for us to grow among this audience. I believe you, you know, bundles are viable. Go to market strategy, adoption will be, uh, maximized with tiered offerings. Tell me more about the tiered offerings. And so then I literally have to just click on the tiered offering hyperlink, and now we're on the slide and having a conversation about that, right?

We don't need to go through the 30 slides. That got me to the fact that I'm recommending a tiered offering. We can just pop over there and we can see what the recommended tiers look like and how that falls out and, you know, have a conversation, right? Because the more conversations you can have, the more you'll, be felt like you're a partner.

It's just. Plus one for making your presentations collaborative instead of a readout. Okay. And just one more here. Again, if you've got the deck, uh, you'll be able to see all of these, but we've got the, the three main bullets on the left addressing the, the main business question. And then based on those three findings, we have a few other suggestions in terms of actions that the client could take to ensure.

Uh, in this one, they're trying to test whether or not they should launch product B. Okay, so hopefully you saw from those TLDR slides that it's important to take a strong stance, be direct, and encourage action to be taken based on the results. Hey, launch at 1499 to maximize revenue or do not sunset, product A.

Product B launched alongside, it captures an, an additional group of the market without cannibalizing, et cetera, right? Because. Full circle moment, right? The best data means nothing if it doesn't inspire action. All right, uh, in the last five minutes, I'm just gonna take you through our typical deck outline to kind of nicely put a bow on this and, and package it together for you.

So what does it look like to partner with Nuer and have a conjoint study that, delivers on all five of the pillars that we've been talking about for the last 40 minutes? Okay, so this is our typical deck outline. I was not lying when I said title slide research context, TLDR, right? Title slide study overview, which is the key questions we set out to answer.

Let me align everyone in the room on why we are here and what we're hoping to answer. And let me remind you who we talk to help us. Figure out the answer, then we'll immediately share our findings to inspire action because people are likely to pay attention at the beginning of a presentation. So you hit them with the good stuff in the beginning so that they stay until the end.

So hopefully I hit you with enough good stuff at the beginning that most of you are, are still here. But imagine you're in this seat, you are presenting your conjoint results. You hit 'em with the TLDR, and they're really excited, and you don't even have to go into the rest of the slides, right? But if you do, or if you just wanna hand over the deliverable, the next thing that we would do after we've inspired action is we'll ground them in their reality.

So we'll do a quick view of the market landscape. And how big the opportunity actually is because when we run a conjoint study, there are other questions besides the conjoint, right? So usually we're, um, gonna include some sort of market sizing element if we can. Very, very simple to do if you're running a B2C study.

A little bit harder if you're running B2B or need to use some sort of targeting. But I love, a great market landscape slide so that we can understand who we're talking to in our survey, how they fall out, what size of the total market they represent, right? Because all of those conjoint simulations are going to be relative to the size of the market that you talk to.

So if you talk to the whole population, then you know, 13% simulation share of pre preference is pretty good. But if you're talking to 2% of the population, 13% of the share of preference is teeny, teeny tiny, right? And it's all relative, and they need to understand. That baseline. So I love a good market landscape slide or even just a, Hey, we talked to these people and they represent this size of the market.

So everything you're gonna see in the conjoint is relative to these 10 million people or whoever, or, you know, 30 million business decision makers in addition to the landscape will also likely have some slides on brand the purchase funnel, right? Brand awareness consideration, favorite, never buy, so that we can really level set the conjoint data because conjoint, while wonderful assumes a hundred percent awareness of our product, equal market marketing and sales efforts, all, all of the best things, right?

And so conjoint results are typically best case scenario. And for some of our clients the vast majority of consumers are aware of their brands, but for other clients, awareness is quite low. And so, you know, they're only gonna see those shares in the conjoint if they really put their money. In marketing and sales.

Okay? So we really like this kind of landscape to level set, you know, what are the external factors that you also might be up against in order to, you know, see the shares that you're gonna see in the contract results. Then we move right into those simulations, so then I can connect, right? Here's the size of the market and here's what you're going to get.

Okay? And they can see the data behind those recommendations in the TLDR. I don't wanna hide it any further than I have to, and you've seen this before, that high level simulation story. And then I like to tell them, Hey, you know, what is influencing their decisions to buy or not buy product B? Right?

Because maybe they're surprised by how little share Product B was getting right. Or how much share that orange brand was getting. And we'll do that by looking at impact scores derived from the sensitivity. And here, let's say we learn that, you know, brand and price are the two most important things. And inevitably we can't change our brand, right?

So what can we change price. And therefore we look at demand curves right away. Hey, people are choosing based on this, if we can change our price for product B, here's how we should change it. And perhaps they're, you know, particularly interested in that willingness to pay by customers versus prospects.

So I'm gonna show them, you know, two demand curves with revenue behind it. And even better, if our client gave me costing information, I could make recommendations based on profitability. But sometimes we aren't able to change our price or maybe we don't want to. So the next logical question is, if we can't change the price, what are our other options?

So we show them a sensitivity analysis, right? And we have our call outs. Here's where you would see a lift with members, but not with prospects. Here's something that would be detrimental to drop. Please don't drop this. Right? And my little how to reads and all the things. Hopefully you can start to see the, the storyline that's keeping your stakeholder engaged and giving them very clear direction on what they need to do when they walk out of this room.

Just as an aside, you could also show that previous sensitivity chart as a willingness to pay chart. It effectively communicates the same thing. If you've done the analysis the way we do the analysis, but it translates that data into shares or that data into dollars instead of shares. Depending on my clients, I'm gonna choose one and put it in the deck and maybe put the other one in the appendix.

Okay? They don't need to see two slides with two different numbers that effectively communicate the same thing. Then they get confused. Why does this say this and that, say that, right? Let's just show 'em one, get the buy-in. And move on to the discussion, right? Hey, given what you saw, like what do you think you're gonna do next with the data?

What are some other scenarios you might wanna see? Like, is my recommendation for, you know, revenue maximizing even possible? Or you know, does it, are you not gonna make any profit off of that, right? I might even bring up the simulator tool and we might war game. What if we did this? What if we did that right?

Because sensitivity analysis is just changing one thing at a time, not two. And maybe they wanna add feature four and feature five, and what does that do? Right? So I find having time for discussion and war gaming and collaboration allows you to become more of a strategic partner than just a vendor.

And then just to show you, we always have an appendix. And in that appendix slide is probably just like our, you know, standard, what conjoint is, what conjoint isn't, maybe the attributes and levels. We tested some screenshots of the exercise, any assumptions we made in the model or adjustments that we needed to make based on, you know, awareness and distribution.

And then maybe if my client requests them, I would put utilities in here, but they will be shown as distributions so that you can see the variation in the individuals, not just that one boring aggregate value, right? Because one lies are in the average. And two, the whole reason we ran a hierarchical Bayesian multinomial logistic regression model is so that we could capture the heterogeneity of the individuals, right?

So if we just show one, one number. That's so sad. Let's show the distribution. You can see here like super clearly who's brand F and why are people like not on the same page with brand F, right? But who's brand C? And everybody you know is kind of in the same opinion on brand C, right? Really, really interesting data.

And look at the one ordered attribute, right? Which we can see that there's some people who really, really do not prefer nine 60 P and really, really do prefer 4K, right? Maybe there's some interesting value in that and maybe this is flagging to me, I should do a, a clustering analysis, right? So stick those utility plots in the back, show the distributions, and you'll be better prepared for conjoint.

So what's next on your path to confident decision making? Hopefully you've seen how the right framework can transform your conjoint presentations from confusing to compelling, right? So now it's time for you to take that step. Uh, so this is just a little gut check cheat sheet to help you hold yourself accountable.

You know, after you run the deck, run through these questions, you feel confident checking each of these box boxes, right? If you do, then you're probably really well prepared. And also if you like what you've seen here, we're trying to do this across every quantitative methodology, and we're launching the Nuer way, which isn't just about running better quant studies, but really becoming that go-to strategic partner.

So we are launching that next month. If you click on that QR code, not only will you get the deck, but you could also, uh, register to become part of the wait list. And if you're on the wait list, we're gonna do a founding 50 members get 50% off the whole training module and some goodies and all that sort of stuff.

But only if you sign up for the wait list. So if you like the trainings that I give and you're curious to learn more, not just about conjoint, but about all the quantitative methodologies, highly recommend you, you join the wait list. And if not, just go grab the deck.

So Brian, come back and, uh, I was ignoring the q and a,

Brian McEwan: so, no, that's, that's working as intended. We gotta give a, a shout out to Keith. Keith heads up our. Consulting team here at Sawtooth. Just like Megan, a wonderful human, very willing to share and teach. Keith is a phenomenal teacher as well.

So thanks again, Megan. For everyone else reminders, uh, you will, uh, get a link to the recording of this webinar. That will have the QR code. We also did put in the chat, we'll probably put on the, the webinar page a link to. Numer website that'll have a the opportunity to download these slides and check us out, sawtooth.com/events for all our upcoming things.

Megan Peitz: Thank you so much. Thank you for having me, Brian. Thanks everybody for hanging in there.