Call centre QA scorecards: writing yes/no checks that hold up

Most call centre QA scorecards were designed to be filled in by a person with a pen: a list of items, a score from 1 to 5 for each, a weighting, a total. They look precise, but two reviewers scoring the same call often disagree. This article explains why that happens and how to rewrite a scorecard as plain yes/no checks that give the same answer whoever, or whatever, reads the call.

Why weighted scorecards drift

A classic scorecard asks a reviewer to judge. "Rate the agent's empathy from 1 to 5." "Was the greeting professional?" "Did the agent show product knowledge?" Each item hides several decisions, and different reviewers make them differently.

Scales invite interpretation

What separates a 3 from a 4 on empathy? One reviewer gives a 4 because the agent apologised, another a 3 because the apology sounded rushed. Neither is wrong, because the item never said what a 4 looks like.

Adjectives carry the weight

Words like "professional", "clear", "appropriate" and "confident" do most of the work on many scorecards. They describe an impression, not something that happened. An agent who fails "professional" has no idea what to change.

Weightings hide the failure that matters

When items are weighted and summed, a missed mandatory disclosure can disappear inside a score of 86%. The total looks fine, the one thing that could cause a complaint or a regulatory problem goes unnoticed, and coaching focuses on the wrong item.

Reviewers get tired

Scoring the twentieth call of the day is not the same as scoring the first, and with small samples one tired afternoon moves an agent's score. We cover the sampling problem in Call QA beyond sampling.

From scorecard items to observable questions

The fix is to stop asking reviewers to rate and start asking them to observe. A yes/no check asks whether something specific happened on the call, and a yes can point to the moment it happened. This changes three things:

• Agreement goes up. Reviewers can still disagree, but about a fact that is quick to settle.
• Feedback becomes actionable. "You did not confirm the customer's callback number" is something an agent can fix on the next call. "Communication: 3 out of 5" is not.
• Critical items stay visible. A failed disclosure check is a failed disclosure check. It is not averaged away.

Rules for writing good checks

Good checks follow a few simple rules. If a draft check breaks one of them, rewrite it.

One behaviour per check

"Did the agent greet the customer, state their name and confirm the reason for the call?" is three checks. If one part is missing, nobody knows which. Split it.

Observable in what was said

A check should be answerable from the words on the call. "Did the agent sound friendly?" depends on tone and impression. "Did the agent thank the customer before ending the call?" depends on words. Prefer the second kind.

Answerable from the call alone

If answering needs the order system or knowledge of what happened next week, the call cannot settle it. "Was the quoted price correct?" needs the price list. "Did the agent state the total price including VAT?" does not.

No adjectives

Replace words like "clear", "proper" and "appropriate" with what you would actually expect to hear. If you cannot say what that is, the check is not ready.

Phrase it so that yes is the good outcome

Mixed phrasing confuses reviewers and reports. A yes should always mean the agent did what you wanted.

Eight scorecard items, rewritten

The table below takes typical scorecard items and turns them into checks. Some items become more than one check, which is normal.

Notice that "Compliant call" becomes several narrow checks, each tied to one requirement. Which disclosures apply to your calls depends on your sector and your legal advice, so write the checks from your own requirements rather than from a generic list.

Grouping checks per call type

A cancellation call needs different questions from a new sale or a complaint, and asking all of them everywhere fills reports with noise. Group checks by kind of call instead. In VOIX these groups are called lenses: a "Complaint" lens might hold acknowledgement, apology and next-step checks, while a "Cancellation" lens holds retention and confirmation checks. A lens can run on every call or only on the calls it fits, so the right checks run on the right calls. A few checks, such as the recording notice, usually belong on every call.

Not applicable versus not answered

Two situations look similar and should be kept apart.

• Not applicable means the check does not belong on this call at all. A cancellation-terms check on a call where nothing was sold is not a fail. Handle this through grouping: the check should not run on that kind of call.
• Not answered means the check does belong, but the call does not settle it. The recording cuts off, the relevant part was inaudible, or the conversation never got to the point where the behaviour would be expected.

An honest "not answered" beats a forced result. A check that is often left unanswered either depends on something outside the call or runs on calls where it does not apply.

Calibrating with reviewers

Before new checks go live, run a short calibration.

1. Pick a small set of real calls covering the call types you care about, including a few difficult ones.
2. Have two or three reviewers answer every check independently. No discussion first.
3. Compare answers check by check. Where reviewers split, the cause is usually an ambiguous word or two behaviours in one question.
4. Rewrite and repeat until disagreements are rare.
5. Write down edge cases. If "confirm two identification details" excludes the customer volunteering them unprompted, say so in the check.

Reviewing failed results with evidence

A failed check is the start of a conversation. Tie every result to the moment in the call that supports it: with a quote attached, a team lead can confirm a fail in seconds.

When reviewing fails:

• Read the evidence before coaching. If the quote does not support the result, the check probably needs rewording.
• Look for patterns. The same fail across a week is a habit worth coaching.
• Separate agent issues from process issues. If a whole team fails a check, the script, the training or the check itself may be the problem.

For service teams, failed checks often point at broken promises to customers. See Customer service follow-up promises for how to track those.

How VOIX helps

VOIX checks every recorded call against your own plain-language yes/no questions, and every answer comes with evidence from the call. Checks can be set for the whole organisation or per team, and AI can draft checks from a description for you to review before they go live. Lenses group checks for each kind of call, so the right checks run on the right calls. Results only count what the call shows, not tone of voice. Read more about checks and lenses or see how this works for customer service teams.

This article is general information, not legal advice. Check the rules that apply to your organisation with your legal adviser or data protection officer.

New to a term? See the glossary.

VOIX for customer service teamsCatch every promise, complaint and callback.