Skip to content
ChatterfyHelp Center
HomeGeneralCRMAI ManagerTrackerPlatformsFAQUpdate log
FAQ
Tracker Errors
Telegram Errors
FAQs
Frequently Asked Questions
The bot stopped accepting channel join requests — why?
Do I have to keep my balance above $100?
AI takes a long time to respond / works slowly
Channel not visible when creating a channel campaign — how to fix it?
I imported a flow, but the bot is still following the old funnel
Why the AI Doesn't Skip a Step If the Lead Has Already Answered Appropriately
How to make the bot wait for a lead's reply
How to remove the keyboard (reply keyboard) for a lead
How do I restart a funnel for a lead?
Requirements for Uploading a Custom Landing Page
Uneven A/B testing — 90% to one branch, 10% to the other
Voice message without visualization (waveform)
Why conversations and events are not tracked — only Subscribe
How to Send a Round Video Message (Video Note) Correctly
How to avoid video compression when sending videos
How to hide chats between agents
  1. FAQ
  2. FAQs
  3. Uneven A/B testing — 90% to one branch, 10% to the other

Uneven A/B testing — 90% to one branch, 10% to the other

Set a 50/50 distribution in or Assign Manager 50/50 for two operators — but you see 90% of leads going one way and 10% the other. This is not a bug or a misconfiguration — it is statistically normal f

Updated June 21, 20262 min
Set a 50/50 distribution in A/B Router — Random Distribution or Assign Manager 50/50 for two operators — but you see 90% of leads going one way and 10% the other. This is not a bug or a misconfiguration — it is statistically normal for small samples.


Why this happens

Any random distributor works like flipping a coin for each lead. Over thousands of flips, the ratio evens out to 50/50. Over a dozen, it may turn out to be 7/3, 8/2, or even 10/0. This does not mean “the coin is biased” — it means “there have not been enough flips.”
In simple terms: to reliably display a 50/50 distribution, you need at least 200–500 leads through the A/B step. With 20 leads, you may see 14/6 (a “70/30” ratio), and this will be within the normal range. With 1000 leads, a difference between 50/50 and 51/49 is already informative.
The same applies to Assign Manager
If Assign Manager — chat distribution is set to 50/50 for two operators, but one received 8 chats out of 10 — with a sample size of 10, this is normal. After 200 chats, the difference will even out. If an operator complains about overload, let 100+ leads pass through the distribution before changing the settings.


How to check that everything is okay

  1. 1

    Open analytics

    For A/B Router — Steps Conversion — funnel by flow steps (you can see how many passed through each branch). For Assign Manager — Chats — conversation analytics (chat distribution among operators).
  2. 2

    Wait for enough volume

    Wait until at least 200 leads have passed through A/B Router. If you get 30 leads per day, that is a week of waiting. Until then, do not draw conclusions about a “broken” distribution.
  3. 3

    Compare the numbers

    If, with 200+ leads, the distribution differs significantly from the configured one (for example, you set 50/50 but got 70/30), then it makes sense to take a closer look at the config. Until then, it is statistics, not a bug.
To build an A/B sample faster
Test hypotheses where there is a lot of traffic. In cold tests with 5 leads per day, any A/B will be “unbalanced” — you will spend a week and not get an answer. If there is a new launch with a large flow, it is better to test hypotheses there.


FAQ

Then there is reason to suspect that something is wrong with the settings. Check: whether the percentages are specified correctly in A/B Router — Random Distribution (often a typo in “50” turns it into “5”), whether there are any conditions before the A/B step that filter out some leads before distribution, and whether there is a filter on one of the branches.

There is currently no strictly “fair” mode (round-robin 1-1-1-1) in the standard A/B Router. With A/B Router — Random Distribution you can only make it probabilistic. If you need exact alternation, implement it via Custom Field (counter) + Condition — conditional branching.

A/B Router within a single flow distributes leads across two branches of that flow. If you want to distribute them between different bots, that is a task for Sources — Traffic Sources (multiple Sources with different links, each leading to its own bot). This is a different mechanism.

A/B Router supports multiple branches—not just 2. You can create 4 variants at 25% each, 3 at 33%, etc. But the more branches there are, the more volume you need for a meaningful result.


Related

A/B Router — Random Distribution
The A/B Router step randomly distributes leads across different branches of the scenario in a specified percentage ratio . Use it for A/B tests: run two message variants, compare conversion, and keep
Assign Manager — chat distribution
The Bot Settings → CRM defaults → Assign Manager section controls which chats each operator sees in . This is a basic team discipline setting: with the right configuration, everyone sees only the chat
Steps Conversion — funnel by flow steps
Steps Conversion is a tool that shows how users go through your scenario step by step. You select a sequence of steps in a flow (for example: Welcome → Qualify → Offer → FD), and Chatterfy shows: how
Chats — conversation analytics
The Chats tab in the — is the main analytics view for the number of bot conversations. Here you can see: how many users messaged the bot how activity changes by day whether leads return how effectivel

Was this article helpful?

BackRequirements for Uploading a Custom Landing PageNextГолосовое сообщение без визуализации (waveform)

On this page

  • Why this happens
  • How to check that everything is okay
  • FAQ
  • Related