Closed means it worked: the Proof Standard v1
A loop counts as closed only when the outcome registered at its opening happens, tested on the customer’s own data.
Version 1.0 · 4 November 2026 · 7 min read
Summary
Most AI in customer service reports its own success, on its own definition, in its own dashboard. A conversation is marked resolved when the customer stops replying, or when a ticket closes, or when a model decides it went well. None of those says whether the problem stopped.
This standard defines what it means to prove that a problem was fixed: the terms, four metrics with formulas, the measurement rules, and who grades the result. It is open. Any company can use it to judge its own service, and any vendor, OnClarity included, can be held to it.
The test of the standard is simple: two analysts applying it to the same data get the same numbers.
Purpose and scope
Purpose. To give service leaders, finance teams and procurement a definition of “fixed” they can defend, formulas they can run, and rules on who grades.
In scope. Customer service across every assisted and self-service channel (voice, chat, messaging, email, in-app, social), whether handled by AI, by people or by both.
Words used. “Must” is a requirement. “Should” is strongly recommended; a report that departs from it says why. “May” is optional.
Definitions
Contact. One inbound conversation from a customer, in any channel.
Issue. The underlying problem that caused one or more contacts, independent of channel or wording.
Repeat. A contact from the same customer about the same issue within the window for that loop type, in any channel. A known issue that reaches new customers after its loop opened also counts as repeat demand for that issue.
Loop. Work opened on an issue, with a named owner, a desired outcome, a test and a window.
Desired outcome. The measurable result the loop is opened to achieve, for example “contacts on this issue down 40% against baseline, held for 30 days”. It must be registered when the loop opens, before any fix ships. It must never be chosen after the results are known.
Loop types and windows. Customer loop: 14 days. Issue loop: 30 days. Knowledge loop: 30 days. Quality loop: 30 days. Revenue loop: to the customer’s renewal date.
Baseline. Eight weeks of contacts before the fix, across every channel, adjusted for seasonality, and agreed by finance.
Loop value. Contacts at stake multiplied by cost per contact, plus revenue at risk where that data is shared, multiplied by a severity weight for regulatory or complaint exposure.
High-value loop. A loop whose value is above a threshold set in advance. A high-value loop must open automatically when an issue crosses the threshold; it cannot be opened selectively.
Closed on outcome. A loop whose desired outcome happened and held through its full window, measured on the customer’s own data.
Unmatured loop. A loop whose window has not ended. It is neither a success nor a failure yet.
The four metrics
| Metric | Formula | Unit | Cadence |
|---|---|---|---|
| Time to know | Time from the first customer signal of an issue to the issue being raised with a named owner | Hours; median and 90th percentile, high-value issues | Monthly |
| Repeat cost removed | Sum over loops closed on outcome of (repeat contacts at baseline minus repeat contacts after the fix) x cost per contact | Money, and % of cost to serve | Monthly and quarterly |
| High Value Loop rate | Value of high-value loops closed on outcome / value of all high-value loops whose window has ended | % | Quarterly |
| Recontact within window | Contacts marked resolved that are followed by a repeat within the window / all contacts marked resolved | %, by channel and by AI or human | Monthly |
Recontact within window is the check every resolution claim should sit next to. A resolution rate reported without it is a claim, not a result.
Measurement rules
Every high-value loop is in the denominator. Loops cannot be left out because they went badly.
The outcome is registered before the fix ships. A loop with no registered outcome cannot be reported as closed.
Windows are fixed per loop type. Any change to a window is logged with its date and reason, and earlier results are not restated under the new window.
Minimum volumes. An issue with fewer than 30 baseline contacts in eight weeks is reported as low volume and excluded from repeat cost removed.
Unmatured loops are reported separately. They are never counted as closed.
Channel switching counts. A customer who was “resolved” in chat and calls about the same issue three days later is a repeat.
Merged and split issues. If one issue splits into two, each child keeps the parent’s baseline in proportion to its contacts. If two issues merge, their baselines add. The audit trail records both.
A fix shipped mid-window restarts the window for the loop it affects, and the earlier attempt is recorded as not closed on outcome.
Seasonality. The baseline is adjusted with the same method every period; the method is published with the report.
Automatic closure is not an outcome. A ticket closed by a timer, by customer silence or by a model’s judgement of the conversation is not evidence that the issue was fixed.
Grading and audit
The customer’s data grades, not the vendor’s model. The test is computed from contact records: did the repeats stop, did the outcome happen. An AI may draft a grade, but a named human signs each result.
Someone who did not do the work signs off. For high-value loops, the signer is outside the team that shipped the fix, for example a finance or analytics partner.
The audit trail records when the loop opened, the desired outcome, the owner, each fix event, the test, the window and the result.
Anyone can recompute. Every reported number drills down to the loops, issues and contacts behind it.
How it works
Reporting template
Each period, a report under this standard shows:
| Field | What to show |
|---|---|
| Period and windows used | Dates, and the window for each loop type |
| Time to know | Median and 90th percentile, high-value issues |
| Repeat cost removed | Money and % of cost to serve, with the baseline method |
| High Value Loop rate | %, with the number of loops in the denominator |
| Recontact within window | %, by channel, AI and human side by side |
| Unmatured loops | Count and value, reported separately |
| Exclusions | Low-volume issues and any loop removed, with the reason |
| Signed by | The named person who signed the results |
Out of scope in v1
CSAT and NPS as proof of a fix. They are useful signals; they do not show that a problem stopped.
Containment and deflection rates.
Individual agent performance.
Revenue loops before 12 months of CRM data exist.
Versioning and change requests
This is v1.0. Changes are published with a version number, a date and a reason, and earlier versions stay available. Anyone may propose a change. Version 1.1 is planned for March 2027, after feedback from companies using it.
The standard builds on work that came before it: first contact resolution, recontact checks, and the way the DORA metrics gave software delivery a shared, versioned definition of performance.
Honesty box: where OnClarity stands today
We wrote this standard to hold ourselves to it, and we do not meet all of it yet.
Loop tracking is still being built. Our product does not yet register a desired outcome when work opens on an issue.
Our recontact window is shorter. Our analytics track reopens on the same conversation within 7 days, for AI and human resolutions side by side. This standard asks for 14 to 30 days across every channel.
We do not measure time to know yet.
QA on conversations handled only by AI is not available.
We publish no High Value Loop rate of our own until we can measure it under these rules.
What we meet today: reopen tracking, reported for AI and human resolutions side by side, with every number drilling down to the conversations behind it.
Questions people ask
What is AI resolution rate, and why isn’t it proof?
AI resolution rate is the share of conversations an AI is credited with resolving, usually defined by the vendor. It counts a conversation, not a problem, and often over a short window. Without a recontact check in every channel, it cannot show that the problem stopped.
How do you measure first contact resolution when AI handles the first contact?
Count the AI’s contact as resolved only if the same customer does not come back about the same issue, in any channel, within a fixed window. Report AI and human first contact resolution side by side, on the same window.
How do you balance AI resolution rate and CSAT?
Neither is proof on its own. Pair the resolution rate with recontact within window, and use CSAT as a signal of how the experience felt. If resolution goes up while recontact also goes up, the AI is closing conversations, not fixing problems.
Why can first contact resolution be misleading?
Because the window and the method change the number. One team measured the same month five ways and got results from 92% (agent self-marked) to 70% (no repeat within 30 days). A standard has to fix the window and the method before anyone looks at the result.
About
The Proof Standard is published by OnClarity. It is free to use and adapt, with attribution.
Sources
DORA, A history of DORA’s software delivery metrics, 2 Jan 2026.
ACXPA, First Call Resolution: what it is and how to measure it (the same team measured five ways, 92% to 70%).
ICMI, Call Center Metrics: KPIs, 2003 (first contact resolution definition).
COPC Inc., What immediate steps could we take to improve FCR rates? (business versus customer definitions).
SQM Group, First Call Resolution: metric and operating philosophy, updated 20 Aug 2026.
Bain & Company, Net Promoter 3.0, 18 Oct 2021 (anchoring a self-reported metric in audited data).
