Why does Pathmonk need a minimum 5% control group?

In any A/B testing you need to maintain a control group to measure real impact rather than relying on assumptions or external factors. Pathmonk follows the same principle. Even after you’ve validated results and scaled personalization, the system always requires a minimum of 5% of traffic to be allocated to the control group. This allows us to continuously understand the performance of personalization and ensure the data remains accurate over time.

The control group represents the visitors who do not see Pathmonk’s microexperiences. By comparing their behavior with the group that does, we can identify the real impact of personalization on conversions and engagement. Without this comparison, it would be impossible to tell if changes in performance come from Pathmonk or from external factors like seasonality, traffic fluctuations, or new campaigns.

Depending on your traffic, the ideal split may vary. For high-traffic pages, 95/5 or 90/10 splits usually work well. For lower-traffic pages, you might use 75/25 or 60/40 to ensure the control group still gathers enough data for a valid comparison.

The goal is to keep both groups large enough to reach statistical significance, meaning the difference you see between them is real, not random.

Learn more:

• What is a Pathmonk A/B test
• How to change the A/B distribution
• How to check your Pathmonk results 

“I was seeing a good uplift when running 50/50, but not now”

Sometimes customers notice that once they’ve scaled Pathmonk to 95% of their traffic, the uplift no longer looks as strong as it did during the initial 50/50 test. 

When you scale, the control group can shrink to just 5%. If your traffic volume isn’t very high, that group may be too small to produce statistically valid comparisons. A few random conversions (or lack of them) can swing the numbers and make the uplift look weaker than it really is.

Our recommendation:

• Understand the limitation → accept that the 5% control group can be skewed if overall traffic is low.
• Compare against your historical baseline → look at conversion rates from before Pathmonk was implemented to get a cleaner sense of long-term improvement.
• Go back to a 50/50 split → if you want another rock-solid measurement, rerun Pathmonk at 50/50 for a set period. This restores a large enough control group to validate uplift with confidence.

In other words, the uplift is still there, but the math requires either more data or a bigger control group to make it visible again.