What your eval is really measuring.

Suppose an agent produces an answer that is factually correct but leaves the user unsure what to do next. Should it pass? What about a helpful recommendation that sounds more certain than its evidence allows?

Those questions force a product team to be specific about the experience it wants to create. Correctness matters, but so do usefulness, grounding, and the cost of a mistake in the particular workflow.

I like to begin with examples people can disagree about. Put a small set of outputs in front of the people who will use or review them. Ask what they would accept, what they would change, and what would stop them from using an answer.

Disagreement is informative. One reviewer may prize brevity while another needs the evidence spelled out. Before turning those judgments into labels, the team needs to decide which reader and which situation the system is serving.

Then separate the dimensions. An answer can be grounded and unhelpful. It can follow the requested format and still miss the point. Keeping these judgments visible makes it easier to identify what changed when a new prompt improves one dimension and damages another.

An evaluation set should evolve as the team learns. Keep the difficult examples, record why the judgment changed, and include cases where the appropriate response is to ask a question or stop. The useful outcome is a clearer shared understanding of what the product should do.