Engineering Trust and Predictability in a Virtual Agent

Level AI explain how to engineer trust and predictability in a virtual agent through an automated evaluation framework.

In a world where software is no longer a rigid script but a fluid conversation, the traditional QA checklist is officially obsolete.

Evaluating an AI system is fundamentally different from testing a traditional software product for four critical reasons:

• Non-determinism: Leading to different outcomes for the same set of user inputs.
• Hallucinations: Confidently generating false information, posing reputational risks.
• Adversarial Users: Guardrailing is non-negotiable as users actively try to perform unauthorized actions.
• Real World Noise: Ability to simulate and perform across variations in accent, emotions, environmental noise.

The Result? No one can afford to deploy an AI system without rigorous validation.

That’s why Level AI developed ‘Automated Evals’ – a framework that stress-tests an AI system in realistic scenarios that only occur when multiple complex variables collide at once.

This evaluation framework comprises three components – Scenario Generation, Simulation, and Evaluation, that operate in a sequential manner.

In this blog, we will cover each one in detail.

Scenario Generation – Effectively Mimicking Realistic Users

To build a bot that survives production, creating diverse sets of test scenarios that go beyond simple interactions is vital.

If only the “happy path” is tested, where users ask simple questions – evaluation will be nothing more than a vanity metric. It generates scenarios using a combination of:

• Core governance guidelines, instructions and skills deployed during the configuration of the agent
• Knowledge documents and policies attached to the agent
• User environment variations such as interruptions, noise, talking speed etc.

While simulating complex queries from your knowledge base, it doesn’t just use isolated data points. Instead, fetch all related documents and generate queries that may require cross-document processing to come up with an appropriate response.

This helps in ensuring that the virtual agent can synthesize information across multiple sources without hallucinating connections that don’t exist.

Simulation Engine – From Scenarios to Real Conversations

Based on scenarios identified in the previous step, it simulates human-like conversations with the AI.

This is a multi-turn dialogue, allowing it to test the AI’s ability to maintain context, recover from errors, and achieve complex goals.

The simulation injects these variables directly into the test:

• Background Noise: Overlay audio profiles like coffee shops, airports, or busy streets to test the bot’s transcription accuracy and focus.
• Speech Variance: Alter talking speed (words per minute).
• Accents: Rigorously test how the model handles different accents.
• Emotional States: Simulate a variety of user emotions to test agent’s behavioural guardrails.

Role-playing the combinations of these scenarios would take an army of QA testers and months worth of testing efforts.

This simulation framework leverages parallel execution by running high-fidelity conversations simultaneously.

It can simulate a month’s worth of call traffic across a range of scenarios with angry customers, heavy accents, and complex interruptions, in a matter of minutes.

Intelligent Evaluation: The LLM Judge Grading the Bot’s Performance

Instead of relying on human reviewers for QA, it uses LLM as a judge to score the AI’s responses on a range of 0 to 1 across 16 performance parameters.

This helps in ensuring that the evaluations are fast, consistent, and free from human fatigue or bias – all while guaranteeing that the AI’s responses meet your quality standards and. The evaluation parameters can be grouped across following buckets:

1. Response Correctness:

The evaluation system compares the agent’s response against the verified documents or policies it was supposed to reference. We score the performance for:

• Fact-Checking by verifying that the AI retrieved the specific policy details without any hallucination; and
• Relevance by checking if the agent response addressed the user query specifically

2. Tool Calling Accuracy:

To be effective, any virtual agent needs to perform autonomous actions like looking up order history, booking appointments, processing refunds, etc. For every autonomous action, we examine the backend logs to verify:

• Did the agent call the right tool, and pass on the correct parameters? If the user said, “Book a flight to NYC for next Tuesday,”, our evaluator checks if it triggered the correct tool (e.g., Calling book_flight) and passed the right parameters “NYC” as the destination.
• How did the agent handle failure? If the tool returned an error (e.g., “Seat unavailable”), did the agent convey that gracefully to the user?

3. Response Quality:

The framework scores the quality of responses generated to ensure that your agent strictly performs within your branding guidelines, solves for the user concerns – while also deflecting any inputs that need it to reveal sensitive information, perform unauthorised actions or bring profanity into its responses. Responses are graded for:

• Clarity & Conciseness: Is the answer easy to understand, or is it a wall of text?
• Empathy: Did the agent maintain empathy during the conversation?

Conclusion: The Complete Loop

You don’t have to guess if your AI is ready for production. You don’t have to wait for a customer to complain about a hallucination.

Combining Advanced Scenario Generation, High-Fidelity Simulation, and Automated Evaluation, provides a comprehensive suite for evaluating the performance of your virtual agent.

Reviewed by: Megan Jones