Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Quota Management in different test groups without loosing respondents

Survey is with four test groups (showing different concepts after screening questions) and same quota for each test group;
Also I like to conduct the interviews of the four test groups randomly.
So I create a variable TG (test group) with SYSRAND

[% Begin Unverified Perl
my $rand = SYSRAND(1, 4);
if($rand == 1)
    SETVALUE("TG", "P");

and make quota questions with quota of the four test groups f.ex. for age.

If I set a value for TG, f.ex. "P" and quota is full, I loose this respondents. But maybe this quota is not closed in TG "Q".
How could I fix a randomly variable for the test group and could change the "TG" if quota is full in this test group.

Thanks a lot!
asked Aug 8, 2017 by Iris (175 points)

1 Answer

+1 vote
Best answer
You can setup a quota question where you define all 4 test groups.

1-Test group 1
2-Test group 2
3-Test group 3
4-Test group 4

Select "Always Qualify" for each test group.

Under the advanced tab for your quota question, look to the bottom where you will find "Check for Cell Membership". Select "Randomize All".

The behaviour will be to randomly select one "open quota" test group.

This quota question will save the value of the test group selected. You can use skip logic to operate off the quota question also.
answered Aug 8, 2017 by Paul Moon Platinum (68,630 points)
selected Aug 9, 2017 by Iris
Paul, maybe you can help me again.
I work now with this quota question.
Next step is that I have 2 groups/quota for each of these test groups and within these groups quota for age, sex and usage.
If I set the quota "Test group" it is fixed and if the f.ex. lower age group of test group 1 is full I cannot switch this person to test group2 - can I?
If you need to introduce other variables into the quota criteria, you can define your quota cells with those additional variables and you use the same approach.

So instead of defining just test group 1-2-3-4, you may have something like the following ...

1-Test group 1 18-49 Male
2-Test group 1 50+ Male
3-Test group 1 18-49 Female
4-Test group 1 50+ Female

5-Test group 2 18-49 Male
6-Test group 2 50+ Male
7-Test group 2 18-49 Female
8-Test group 2 50+ Female

and so on ...

For all open quota cells, one will be randomly selected.
But if I do so, I think I must define cross quota which I do not like to define.
One quota cell example could be:
1-TG1 user1 18-29 male
2-TG1 user1 30-39 male etc.
this would be 16 groups per test group (2user 4ages 2sex).
But I only want quota for
TG1 user1
TG1 user2
TG1 18-29
TG1 30-39
TG1 40-49
TG1 50+
TG1 male
TG1 female
and if one of these cells for TG1 is closed use TG2, 3 or 4...
You need to be careful about the allocation process because without cross quota, you could select test group 1 and user and age are not filled for this test group but their gender is.

Once user, age and gender have been answered, you should have quotas for these up front, without test group. If their combination of user, age and gender are filled, then they terminate. No point going on regardless of the test group.

Once you know their user, age and gender combination is not filled, you can then allocate them to 3 quota questions ...

Test group + user (random selection based on user)
Test group + age (conditions can be based on the test group selected at Test group + user and the age question)
Test group + gender (conditions can be based on the test group selected at Test group + user and the gender question)

And I always ask myself - how strict are the quotas? You can fiddle with quota targets during field via the admin module and opening up some quotas that are nearing their targets.
I am sorry, but I don't understand this. It is clear that I use quota questions for age, gender, users separately and before fix the test group.
But the next steps are not clear.
How looks the quota cells in "Test group + user" when I not define test group before. And are they ramdomly (like test group before) ?
Thanks a lot!!
Once user, age and gender have been answered and they have been allocated to their individual quotas for these variables, you can then move on to the test group problem. At this stage, if any of user, age and gender were filled, the respondent would have terminated the survey.

If you create a quota question which represents test group and user only, it would look like this ...

Call it QTTGUser.

TG1 user1
TG1 user2
TG2 user1
TG2 user2
TG3 user1
TG3 user2
TG4 user1
TG4 user2

Define all the user 1 quota cells using the user 1 definition only.
Define all the user 2 quota cells using the user 2 definition only.
Set the Cell  Membership to randomize all.
So a test group will be selected randomly for a open user quota.

Now setup a quota question which represented test group and gender only, it would look like this ...

Call it QTTGGender.

TG1 male
TG1 female
TG2 male
TG2 female
TG3 male
TG3 female
TG4 male
TG4 female

The Cell Member ship would be set to Sequential.
Each of the quota definitions for test group / gender would be defined with the logic: QTTGUser and Gender

e.g. TG1 Male: QTTGUser=1 or 2 and Gender=M.

Setup the test group / age the same way you setup the test group / gender. Call it QTTGAge.

So in summary, you have conducted these steps ...

1/ Established user and allocated to the user quota if open, else terminate.
2/ Established gender and allocated to the gender quota if open, else terminate.
3/ Established age and allocated to the age quota if open, else terminate.
4/ Randomly allocated an open test group within the QTTGUser quota question and recorded the Test group / User.
5/ Recorded the Test group / Gender in  QTTGGender.
6/ Recorded the Test group / Age in  QTTGAge.

As mentioned earlier, you can fiddle with quota targets and open up certain targets during field if you want via the admin module.

Does that help?
Good Morning, Paul and thanks a lot!
But also in this casse, if I set the variable test group for one resp. to "1" (in this case with the quota question: QTTGUser) and gender male is filled, these respondents will get lost - also gender male is not filled in TG2/3/4. Right?
So there does not exist a solution for that problem - I have to watch the data and react immediately..?
That's why I was saying earlier that not applying the cross quota method can create the situation you mentioned.

Depending on how strict your quotas are, you could either apply the cross quota method (I know this involves a lot more quota definitions) or you keep an eye on survey progress and open up quotas where necessary.

If you have some wriggle room with your quotas, you can set them a little higher for QTTGUser, QTTGGender and QTTGAge so none quota fail.