LLM evaluation

An evaluation set is a fixed sample of real conversations used to score an AI agent’s quality before and after changes. Holding the sample constant is what makes results comparable, so evaluation sets are versioned and refreshed only deliberately when the underlying contact mix genuinely shifts.

A large language model is the model class behind modern conversational AI, summarisation, and reply drafting. Trained on broad text corpora, it predicts language fluently but holds no inherent knowledge of a specific business, which is why grounding and retrieval are required before answering customer questions.

A hallucination is a fluent but unsupported answer produced when a model is not grounded in real data. In customer service the risk is specific: an invented policy, price, or delivery date reads as authoritative to the customer and creates a commitment the business never made.

Grounding is the practice of constraining an AI agent’s answers to approved knowledge sources and live system data rather than to the model’s own training. Grounding is the main defence against hallucination, and it is what makes an AI answer auditable back to a specific document or record.