Glossary
/
Performance

Accuracy Threshold

The minimum score an AI agent must hit in testing before it goes live.

Definition

An accuracy threshold is the minimum score an AI agent must reach in testing before it's allowed to go live. The business sets the bar based on the risk of each task, and the agent is scored with evals on a set of real customer questions from the company's own data.

Key points

<ul><li>It answers the first question a CIO asks: "How often will it get the order wrong?"</li><li>Tasks that move money or stock, like booking orders, usually need a higher bar than answering product questions.</li><li>The ontology and knowledge graph reduce errors like picking the wrong SKU. Evals block go-live below the threshold.</li><li>A head-to-head test on the same set of questions is also a fair way to compare vendors.</li></ul>