Improving an AI agent by changing only the prompt

1

The benchmark

In the benchmark, the agent helps a simulated customer, uses tools that change bookings and follows a policy. A task succeeds when it communicates the required information and leaves the database in the expected state.

I kept the model and tools fixed to isolate the effect of prompt changes.

benchmark setup: simulated customer, agent, tools and policy

2

Beyond the final score

The default agent solves 34 of 50 tasks. That binary score, however, cannot distinguish a correct answer without an action from a badly executed action.

I therefore examined Langfuse traces and separated communication, final database state and tool actions. Each error type points to a different fix.

break down a binary score into the information needed

3

Three error types

The traces show three families: the agent describes an action without performing it, performs an unauthorized action, or picks the first valid option without comparing alternatives.

These distinctions guided the prompt changes. The figures show a sample trace and how I classified the errors.

Langfuse trace of a failed conversation, reward 0.00

a real trace: the conversation step by step and why it scored zero · open full screen ↗

three failure types found in the traces

4

When added rules worsen the result

In three versions, I added instructions for observed cases. Scores did not improve; the chart shows why I changed approach.

In the final prompt, I used more selective instructions and removed an example that led the agent to add an unrequested service. The result applies to this model and these tasks, not every agent.

How to read the chart Each intermediate version has one run; the final version was repeated four times. The final results compare the averages.

six versions during development; the final version was then repeated

5

Revising the prompt

I removed an example that confused a customer entitlement with an actual request. I then made two steps explicit: compare options when the customer asks for the cheapest, and say a change has not yet been made before asking for confirmation.

three prompt lines as the agent sees them

The final prompt is in English. The highlighted lines address the three failure types.

Serve what they asked for, and ONLY that: something the <policy> permits, or that they are entitled to - a free checked bag, a compensation, an upgrade they qualify for - is not something they asked for. An entitlement is not an instruction. If they did not ask for it, do not add it and do not offer it.

When the customer DESCRIBES what they want instead of naming it - "the cheapest", "anywhere on the west coast", "either airport" - that description is a search, not an instruction to take the first thing that fits. Search every combination the description allows, list the candidates you found with the number that decides between them, and choose by comparing those numbers. Write the list out before you choose.

Open that message with exactly this line, on its own:
I have not made any of these changes yet - I need your go-ahead first.
It matters because this message reads like a receipt, and a customer who thinks the work is already done will end the call before you can do it. Nothing is done until you call the tools.

Excerpts from patches/tau2-custom-agent.patch, sections PROCEDURE and CONFIRMATION_FORMAT.

Before each test, I noted which tasks I expected to change. Later runs let me compare those predictions with observed behavior.

1. UNDERSTAND. Name to yourself every distinct thing the customer asked for;
   one message often contains more than one. … Serve what they asked for,
   and ONLY that: … Before you go further, it can help to glance again
   at the parts of the <policy> that cover what you just named - keep them
   in mind for the steps below, rather than trying to recall them from earlier
   in this conversation.

…

3. CHECK ELIGIBILITY. The tools do not enforce the policy: a call can succeed
   and still be wrong. … Once you have an answer, it is worth checking it
   once more against the exact wording of the <policy> lines it rests on, in
   case the reasoning above drifted from what they actually say.

from the final prompt: two lines of guidance for reasoning before acting

6

Result and variation

The final version solved 80.5% of tasks on average across four complete runs. The default agent scored 68% in a single run: a reference point, not an equally stable estimate.

The final version's four scores range from 74% to 86%. Repetition reveals variation a single run would hide.

four identical runs, four different scores

the ten runs in the Langfuse comparison dataset · open full screen ↗