Quality assurance

An evaluation set is a fixed sample of real conversations used to score an AI agent’s quality before and after changes. Holding the sample constant is what makes results comparable, so evaluation sets are versioned and refreshed only deliberately when the underlying contact mix genuinely shifts.

A disposition code is the label an agent or AI applies to a closed contact to describe what actually happened. Disposition data drives volume reporting and root-cause analysis, so inconsistent or overly broad codes quietly destroy a support team’s ability to explain why contacts arrive at all.

Human in the loop describes a review step where a person approves or corrects AI output before it reaches the customer. Teams typically apply it to high-risk intents such as refunds, cancellations, and complaints, then relax it selectively as measured accuracy on those intents proves out.

Model drift is the quiet degradation in AI performance that occurs as products, policies, and customer language change while the model and its knowledge stay fixed. Drift rarely announces itself; it appears as a slow rise in escalations and fallbacks on intents that previously performed well.