Introduction
Randomization plays an important role in reducing bias in surveys. Generally, answers to survey questions should be randomized when they are inherently unordered, and the order of questions should be randomized when appropriate.
Randomization
Response options
Questions that include a list of options will include a randomization setting. You can toggle randomization in the settings section of a question.
When this setting is used, the order of options seen by the respondent is not saved. If you would like to save the order, use dynamic lists.
Questions
To randomize the order of questions on a page:
- Click the options menu icon (
) that appears when hovering on a page in the navigation panel. - Flip the toggle labeled Randomize questions.
Questions will now be randomly ordered for each respondent, and an indicator will be shown next to the page name.
Note: The order of randomized questions is not saved in a respondent’s data record.
Pages
Page randomization is managed at the block level. Every question and page in a survey resides within a block. By default, all questions are added to a single block, but you can organize your survey into sections by placing pages in multiple blocks. Learn more about this in the Blocks article.
To randomize pages:
- Hover over a block in the navigation panel and click the options menu icon ().
- Flip the toggle labeled Randomize pages.
Once enabled, pages within the block will be presented in random order for each respondent. An indicator will appear next to the block name to show that randomization is active.
Note: The order of randomized pages is not saved in a respondent’s data record.
Blocks
After your survey contains multiple question blocks, they can be grouped into a Block collection and randomized. To randomize blocks in a collection:
- Hover over a collection in the navigation panel.
- Click the options menu icon ().
- Toggle on the Randomize blocks option.
Alternatively, the toggle is available in the middle editor section when the collection is selected.
Once enabled, blocks within the collection will be presented in random order for each respondent. An indicator will appear next to the collection name to show that randomization is active.
Note: The order of randomized blocks is not saved in a respondent’s data record.
Survey randomization and skip logic
Randomizing sections of a survey can cause unique interactions. Learn how randomization affects different areas of a survey.
Questions on a page
Consider a page set to randomize four questions:
Page 2
Q4
Q5
Q6
Q7
If you create skip logic before page 2 that skips to Q6, only questions Q6 and Q7 will be randomized and displayed.
Pages in a block
Consider a block with 4 pages set to be randomized:
Page 1
Page 2
Page 3
Page 4
Similar to questions on a page, a skip to an element on Page 3 will result in Page 3 and Page 4 being randomized.
Blocks in a collection
Consider a block with 4 blocks:
Block 1
Block 2
Block 3
Block 4
Collections work slightly differently. The block order is determined first, and then the skip is executed. For example, if the collection resulted in an order of:
Block 4
Block 1
Block 3
Block 2
and a skip sent the respondent to Block 3; they would only see Block 3 and Block 2
Subset of blocks in a collection
If a block subset is used, the approach described above for collections would apply, but if a block were not included in the subset, the respondent would begin at the start of the collection.
For example, if a subset resulted in
Block 4
Block 1
Block 2
and a skip directed a respondent into Block 3 (which is not in the subset), they would instead begin at Block 4 and see Blocks 1 and 2.