Tines Academy › Lesson

Check quality and cost

Review the agent’s answers, response time, and available cost information, then save checks you can repeat after a change.

Objective: Compare answers with their sources and use available usage evidence to judge whether a change is worth testing.

Check how well it answers

The agent can now answer from policy, look up a user’s own claims, and prepare a summary for finance. You’ve checked those parts as you built them. Now you’ll bring a few of those results together to review answer quality, response time, and cost.

Keep the agent as a draft for this lesson. In the next lesson, you’ll use these results to help decide what’s needed before a team can use it.

Judge the answer, not the exact wording

An agent may phrase the same answer differently each time, so focus on whether the facts, conditions, and next step stay correct. For example, “including taxes” and “with taxes included” mean the same thing. Changing that to “before taxes” changes the policy advice and could lead the user to spend more than the limit allows.

In this lesson, you’ll review answers to a few familiar questions against the policy or claim records. You’ll also save the questions and expected results so you can repeat the checks after future changes.

What makes an answer useful?

Understand what each answer uses

Each model call uses tokens: units of text or other content the model receives and generates. The model, the amount it reads, and the length of its reply affect both cost and how long an answer takes.

The next prompt asks Tines 3B to report the response time, usage, and estimated cost it can measure for a few agent conversations. You’ll compare those results with answer quality and consider whether the wait is reasonable for an expense question. The report should separate the agent’s usage from the AI used to build it and leave unavailable measurements unknown.

You’ll also review any operating limits, such as how long an answer can take or how many tool calls the agent can make, and what users see when a limit is reached.

Review quality and usage

Choose the tab that matches how your agent looks up claims, then send the prompt in workflow chat.

Review three questions using the loaded policy and fictional claims:

- A policy question with a known answer.

- "Why was my claim EX-101 returned?"

- A question about an above-limit expense before booking.

If the fictional policy is loaded, use "What's Berlin's nightly hotel limit?" and "I haven't booked. Can I expense a $260 Berlin hotel night?" for the policy questions. Otherwise, choose equivalent questions supported by the loaded policy.

Reuse earlier results only if the relevant instructions, model, tools, and sources haven't changed. Run missing tests in separate conversations where possible. Show each question, source-based expected answer, actual answer, and references to the supporting sources and test results. Mark anything you couldn't verify as unverified.

Use execution data to report available response times, tool calls, tokens, and estimated model costs. Keep the agent's usage separate from building usage. Explain what each measurement covers and which prices the estimate uses. Don't present totals as the cost of one answer or treat missing measurements as zero.

Identify incorrect answers or unnecessary work. Check any response-time and tool-call limits and what users see when they're reached. Say if a limit isn't set or couldn't be tested. Recommend changes, but don't make them yet.

Save reusable questions, source-based expected answers, the policy version, references to results, and unfinished checks in Readme. Use safe descriptions or fictional examples rather than private information. Apart from this update, don't change the build or push it live.

Review three questions using the approved policy and approved test records belonging to the signed-in test account:

- A policy question with a known answer.

- A question about a claim's recorded return reason.

- A question about an above-limit expense before booking.

Ask me to identify suitable approved test records if needed. Don't change claims or retrieve unrelated records to create test cases. If a suitable record or policy scenario isn't available, mark that check as unverified.

Reuse earlier results only if the relevant instructions, model, tools, and sources haven't changed. Run missing tests in separate conversations where possible. Show each question, source-based expected answer, actual answer, and references to the supporting sources and test results. Mark anything you couldn't verify as unverified.

Use execution data to report available response times, tool calls, tokens, and estimated model costs. Keep the agent's usage separate from building usage. Explain what each measurement covers and which prices the estimate uses. Don't present totals as the cost of one answer or treat missing measurements as zero.

Identify incorrect answers or unnecessary work. Check any response-time and tool-call limits and what users see when they're reached. Say if a limit isn't set or couldn't be tested. Recommend changes, but don't make them yet.

Save reusable question patterns, source-based expected behavior, the policy version, references to results, and unfinished checks in Readme. Don't copy private claims or conversations. Apart from this update, don't change the build or push it live.

Check the quality and usage review

Read the report in workflow chat. If it reuses earlier results, check that it explains why they still apply to the current draft.

Check the answers

Compare each reported answer with the policy or test claim record. Facts, conditions, and next steps should match.

Expected answers should come from those sources, not from an earlier reply by the agent.

Check the response time and cost

Look for:

Measurements for the agent’s answers, separate from building usage.

The runs or time period covered.

The prices used for cost estimates.

Missing measurements marked as unknown, not zero.

Consider whether the wait is reasonable for the task. Look for repeated searches or long replies that don’t help the user.

If you don’t yet have a budget or expected usage level, note that cost still needs assessing before team use.

Check the operating limits

Find any response-time or tool-call limits and what users see when they’re reached. The report should say if a limit isn’t set or hasn’t been tested.

Check the saved questions

Open Readme. Look for reusable questions, expected results, the policy version, references to test results, and unfinished checks. It shouldn’t contain private claims or conversations.

If something doesn’t match

If a result differs from what you expected, or the report is missing evidence, ask Tines 3B to investigate. Ask for a focused fix where needed, keeping source and access checks intact.

After a change, rerun the affected checks. Ask Tines 3B to record anything still unverified in Readme.

What to do if something still feels fuzzy

You reviewed the agent’s answers against their sources and looked at the available response-time, usage, and cost information. You saved reusable checks and noted gaps in the Readme. Next, you’ll decide who looks after the agent and what still needs checking before team use.

Build it safely: You checked the evidence behind the answers and usage report, keeping missing results visible rather than treating them as successful checks.