Proof of concept

Containment rate is the share of automated conversations that end without escalation to a human. It is the primary efficiency measure for AI support, but a high rate only indicates success when paired with satisfaction and repeat-contact data confirming customers actually got their issue resolved.

Escalation rate is the share of contacts transferred to a higher support tier, a supervisor, or from an AI agent to a human. Reporting should separate planned handoffs, where an AI agent routes a complex or high-risk case by design, from failure escalations, where the agent could not resolve the issue; blending the two obscures true automation performance. Escalation reason codes point to specific knowledge base gaps, and the metric moves inversely with containment and first contact resolution. Example: 120 escalations from 3,000 contacts gives a 4% rate.

An evaluation set is a fixed sample of real conversations used to score an AI agent’s quality before and after changes. Holding the sample constant is what makes results comparable, so evaluation sets are versioned and refreshed only deliberately when the underlying contact mix genuinely shifts.

An AI agent is software that resolves a customer request end to end, taking actions in connected systems rather than only answering questions. Unlike a scripted chatbot, an AI agent can look up an order, issue a refund, or update a record, then confirm the completed outcome back to the customer.