Lighthouse Studio

Introduction to Quota Control

Survey researchers often want to close the survey to new respondents after a specific number of completed records, or after a certain number of records are completed by specific types of individuals (e.g. males, females).  This section discusses Quota Control functionality available in Lighthouse Studio for obtaining user-specified numbers of completed records within given classifications.

 

The desire for Quota Control can be economic: respondents completing the survey are paid a cash incentive and the researcher does not want to go over-budget.  For example, we might want to achieve 1000 total completes. Quota Control is also used to achieve a sample reflecting known characteristics of a population: representative proportions of male, female, low income, high income, etc.  Using Quota Control can also help us conserve our relationship with respondents: we limit the amount of time we ask of respondents to a few screener questions if we do not need their completed record.

 

---

Quota Based on a Single Question

 

Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain number of completes within different company sizes (large, small):

 

 

A respondent starts the survey and is first asked what size company she works for.  This is called the screener question.  After the screener question is asked, we determine whether another respondent with this characteristic is needed.  If her Quota Cell is already filled with enough completed records for her company size, then we can skip her to the end and mark her as disqualified.  Otherwise, she is allowed to continue the survey.  This process is represented by the diagram below:

 

 

The Quota Qualification Point is inserted in the questionnaire from the Write Questionnaire dialog, by inserting a quota "question" (in the example above, directly following Q1).  The quota question appears in the question list alongside other questions.  The quota question really isn't a question in the traditional sense (because we don't actually show anything on the screen for this question or ask the respondent to provide an answer to a Quota question).  The Quota question type represents a Quota Qualification Point where we define the quota qualification logic and determine if the respondent belongs to a quota cell that is already full (this determination is made when the page containing the quota question is submitted).  Based on whether a quota is full, we decide at the Quota Qualification Point whether to allow the respondent to continue or send the respondent to another point in the survey (such as a terminate screen).

 

---

Multiple Independent Quotas

 

Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain number of completes within both different company sizes (large, small) and types (manufacturing, services):

 

 

In this case we have two independent quotas: company size and company type.  Within company size, there are two quota cells (large and small).  Within company type, there are also two quota cells (manufacturing and services).  Note that we aren't concerned with (for example) how many people from large manufacturing firms are interviewed (we don't care about the joint occurrences of the two quota variables).

 

A respondent starts the survey and is first asked what size and type company she works for.  These are called screener questions.  After each screener question is asked, we can determine whether this respondent is needed.  If for either company size or company type her Quota Cell is already filled with enough completed records, we can skip her to the end and mark her as disqualified.  Otherwise, she is allowed to continue the survey.  This process is represented by the diagram below:

 

 

In the example above, once we determine that the respondent belongs to a quota cell that is full, we immediately terminate the respondent.

 

---

Quotas Representing the Joint Occurrence of Multiple Variables

 

Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain number of completes within different company sizes (large, small) and types (manufacturing, services):

 

 

As with the previous example, we plan to obtain 700 large company interviews, 300 small company interviews, 500 manufacturing interviews, and 500 services interviews.  But in this example, we are concerned with how many respondents represent each company size AND type (joint occurrence).  To do this, we establish a single quota variable (at a single Quota Qualification Point) with four possible cells (by inserting a Quota question from the Write Questionnaire dialog).  For each cell, we use qualification logic that references both the company size and company type variables using AND logic.

 

The questionnaire flow is shown below:

 

 

 

---

Controlling Total Number of Respondents

 

The simplest application of quota control is to limit the number of total completed records.  From the Write Questionnaire screen, insert a Quota question (preferably on page 1, so that respondents starting a closed survey will immediately be skipped to your termination screen that tells them that the survey is closed).  Add a single quota cell, specify the cell limit (such as 1000, for 1000 total completed records) and check Always qualify.  This tells Lighthouse Studio that all respondents qualify for a single quota cell, irrespective of previous answers, and the maximum number of completed records is 1000.

 

---

Additional Notes and Capabilities

 

Lighthouse Studio's quota control is flexible and powerful.  Quota membership may be based on many variables, resulting in a virtually unlimited number of quota cells.  Screener variables can be "passed in" through a URL link or asked at the beginning of the questionnaire. Quota control cannot guarantee that the quotas you establish for groups are not exceeded.  There may be some overage.  For example, you may wish to interview just 100 males.  A simple rule would be to let males continue the survey if you haven't yet achieved 100 completed male records.  Imagine that 99 males have already completed the survey and five males are currently in progress (have passed the Quota Qualification Point and are in the middle of the survey), but these five haven't yet finished the survey and been counted as complete.  A new male respondent reaches the qualification point and is allowed to take the survey (since only 99 complete records have been recorded).  We can see how this could result in 105 completed records, if all the respondents in progress eventually complete the survey.

 

Lighthouse Studio has more sophisticated rules for determining if the Quota Cell Limit has been or is expected to be reached.  You can specify that Lighthouse Studio should assume that, say, 80% of the respondents who have passed the Qualification Point and are actively taking the survey will eventually finish (i.e. a 20% drop-out rate).  You can also specify that respondents who haven't completed the survey within a certain amount of time should be marked as Inactive.  If these options are in place, when a new respondent reaches the Qualification Point, Lighthouse Studio considers the number of respondents in this respondent's quota cell already completed and actively in progress (discounted by 20%) to decide if the new respondent is needed.  We refer to this ability to determine more intelligently if more respondents are needed within a Quota Cell as Predictive Quota Control.  Predictive quota control can certainly limit the amount of overage, but it still cannot ensure that quota cell limits are not exceeded.

 

Restarts:  When a respondent abandons a survey and returns to take the survey (restarts), if the respondent belongs to a quota that is now filled, the respondent will be immediately skipped to the Disqualify Question associated with that quota (the question specified on the Settings tab).