Variant comparison allows you to directly compare performance metrics between blocks of the same type across different variants. This makes it easy to identify which variant performs better and make data-driven decisions about your designs.
Distribution Metrics
For studies using exclusive comparison, the variant distribution shows the percentage of participants assigned to each variant along with the participant count. This helps you verify participants were distributed evenly across your variants and understand the sample size for each group.
Creating a results comparison
To compare performance between variants, you'll need to create a comparison:
Click “Create new comparison" in the Results comparisons section
Select the blocks you want to compare from each variant
You can only compare blocks of the same type (for example, two prototype tests or two multiple choice questions)
Once created, the comparison will display key metrics side-by-side, making it easy to spot performance differences between variants.
You can create multiple comparisons to analyse different blocks within your variant study, allowing you to evaluate which variant performed better across various tasks and questions. Alternatively you can manually compare results between blocks by navigating between the specific blocks in the block list.
Comparison metrics by block type
Prototype test comparison
Reached the goal screen (Goal based tasks): Percentage of participants who successfully reached the goal screen in each variant
Misclick rate: Average percentage of clicks outside hotspots for each variant
Average duration: Average time participants took to complete the task in each variant
Tree test comparison
Outcome
Direct success: Percentage of participants who reached the correct destination without backtracking
Indirect success: Percentage of participants who reached the correct destination after exploring other paths
Failed: Percentage of Participants who didn't reach the correct destination
Average duration: Average time participants took to complete the task in each variant
Multiple choice question comparison
Answer distribution: Side-by-side bar charts showing the percentage of participants who selected each answer option in each variant
Opinion scale comparison
Average rating: Average rating across all participants for each variant
Card sort / Live website test / Live mobile test comparison
Average duration: Average time participants took to complete the task in each variant
Individual responses
This is where you will find the variant or variant order for each individual participant.
Variant (Exclusive comparison): Which variant the participant completed
Variant order (Alternating comparison): The order that the participant completed the variants
Completion date: The date and time the participant completed the study
