Why reviewing 2% of calls isn't quality assurance

Most call quality assurance still works the way it did when every review meant a person listening to a recording: pick a few calls per agent, score them against a form, and hope they are representative. They usually aren't. This article explains why small samples miss what matters, what changes when every call is checked, and how to write checks that give reliable answers.

How sampling-based QA works, and where it breaks

A typical call centre QA process looks like this: a reviewer picks a handful of calls per agent each month, listens to them, fills in a scorecard and shares feedback in a coaching session. It is a sensible process when listening time is the bottleneck. But it has three structural problems.

Small samples miss rare events

The things QA most needs to catch, such as a missing recording disclosure, a wrong price or an inappropriate promise, are often rare. Rare events and small samples don't mix.

Take a simple, hypothetical example. A team handles 1,000 calls a week and reviews 20 of them, which is 2%. Suppose one call in a hundred contains a serious issue, so ten problem calls a week. The chance that a random sample of 20 contains none of them is about 82% (0.99 to the power of 20). In most weeks, the review finds nothing, and the team concludes all is well.

Per agent it is worse. If each agent gets two or three calls reviewed a month, a single score can swing on one unlucky call, and real patterns stay invisible.

Selection bias creeps in

Samples are rarely random. Reviewers skip calls that are too short, too long, in another language or on a bad line. Calls get picked because a customer complained, which skews the picture the other way. Either way, the sample stops describing the work.

Feedback arrives late

When review happens weeks after the call, the agent barely remembers it. A habit that started on Monday has been repeated on hundreds of calls before anyone mentions it.

What changes with 100% call monitoring

When every call is checked against the same criteria, QA shifts from estimating to measuring.

• Consistent criteria. Every call is asked the same questions. Results don't depend on which reviewer happened to listen, or on how tired they were.
• Evidence, not impressions. Each answer points to what was actually said, so an agent can see exactly why a check failed, and a reviewer can confirm it in seconds.
• Trends you can trust. With complete data you can compare teams and reps, see whether a new script is followed, and spot a drop in one week rather than one quarter.
• Coaching on real calls. Instead of debating whether a sampled call was typical, a team lead can open the three calls where a step was missed this week and talk through them.
• Reviewers spend time where it matters. People stop listening to routine calls that pass every check, and focus on failed or unclear ones.

How to design good checks

Automated checks are only as good as the questions behind them. The best checks share four traits:

1. One question. Each check tests one thing. If you need "and", you probably need two checks.
2. Observable in the call. The answer must be something you could point to in the transcript. Intentions, attitudes and things that happen off the call are out of scope.
3. Yes or no. A check either passed or failed. Save nuance for the coaching conversation.
4. Specific to the call type. A renewal call and a complaint call need different checks. Group them by kind of call.

Good and bad checks compared

The checks on the left feel natural to write, but two reviewers would answer them differently. The checks on the right can be answered from the transcript, with a quote to back the answer up.

Start small and iterate

Begin with five to ten checks for one call type. Run them for a week, read a selection of the results, and refine the wording where answers surprise you. A check that fails often may point to a real problem, or to a question that is phrased too strictly. Either is worth knowing.

When a call doesn't answer the question

Not every call answers every question. A customer may hang up early, the topic may never come up, or the line may be poor. A good QA system says so, rather than forcing a result.

Treat an unanswered check as information:

• Many unanswered results on one check usually mean the check doesn't fit that call type, or that it is worded too narrowly. Move it to a better-suited group of checks or rephrase it.
• Unanswered critical checks deserve human review. If you need to know whether the disclosure was given, an unclear answer is a reason to listen.
• Don't count "not answered" as a pass or a fail. Keep it separate in your scores, so it doesn't inflate or depress results.

Keep humans in the loop

Checking every call does not replace reviewers. It changes their job.

• People decide what good looks like. Checks reflect your policies and your judgement. Review them when scripts, products or rules change.
• People handle consequences. Use check results to find calls worth discussing, not to make automatic decisions about individuals. Article 22 of the GDPR gives people the right not to be subject to decisions based solely on automated processing that produce legal or similarly significant effects on them, so keep a person involved in anything that affects an employee's job.
• People coach. Evidence makes coaching faster and fairer, but the conversation is still between a team lead and an agent.
• Tell your team. Agents should know which checks run on their calls and why. Transparent criteria make the scores easier to accept and act on.

Keep in mind that call recording and analysis must have a lawful basis and be transparent to callers and employees. Our guide to recording sales calls under the GDPR covers this in more detail.

How VOIX helps

VOIX records and transcribes the calls it receives from your phone system and runs your checks on every one of them. Each check is a plain-language yes/no question, and every answer comes with evidence from the call. Checks can be grouped into lenses for one kind of call, set for the whole organisation or per team, and drafted by AI from a description for you to review. Results arrive within minutes of the call ending, alongside a summary, scores and advice per call, and trends by team and rep. See how this works for compliance and QA, customer service and checks and lenses.

This article is general information, not legal advice. Check the rules that apply to your organisation with your legal adviser or data protection officer.

New to a term? See the glossary.

VOIX for compliance and quality assuranceEvery call checked against your script and the rules.