This article is auto-synced from its in-app version in Tai.
Company admins only. The Advisor is visible to users with the Company Admin role.
The Advisor reviews how your agents perform in real use — recorded conversations, report runs, and skill content — and proposes concrete improvements you can review and apply.
When you'd use this
Your agent works, but you want it to work better — and the best evidence is what it actually did. For example: after a busy week, evaluate ten recent conversations of your customer-care agent. The Advisor points out where it dead-ended or misused a tool, and proposes a revised system prompt you can preview as a diff and apply in one click.
Steps
Evaluate an agent's conversations
In the Workbench sidebar, click Agents (under the AI Platform section) and open the agent.
Scroll to the Advisor section below the agent's settings. On editable agents, the Advisor shortcut in the section rail on the left jumps straight there.
Pick up to 10 conversations to evaluate. The list shows conversations from everyone in your company, most recent first — click Load more conversations to page further back.
Optionally add Focus instructions, e.g. "focus on tool usage" or "check the tone with corporate clients".
Click Evaluate n conversations. You land on the run page while the evaluation runs in the background — typically a minute or two. Feel free to navigate away; the run continues.
Review findings and apply proposals
When the run finishes, it shows a summary, findings with links to the exact conversations they came from, and improvement proposals — a rewritten system prompt, skills to attach or detach, or a drafted custom skill. Configuration advice (model choice, attachment hygiene) arrives as findings to act on yourself, not as an applyable proposal.
On a prompt proposal, click Compare before/after to start a comparison run: the Advisor re-reads the evaluated conversations and predicts, turn by turn, what the proposed prompt would have changed — before you adopt anything. It is a prediction, not a re-run: the conversations are not replayed and no tools are called again.
Click Apply to accept a proposal. Applying changes the agent immediately, so review the preview first — nothing is ever applied automatically.
Analyze a report run
In the Workbench sidebar, click Reports and open a run from the run history.
Click Analyze this run. The Advisor checks the output against what the report promises, diagnoses the cause if the run failed or came back empty, and flags wasted reasoning that slows the report down.
Review a skill
Not yet available to your company. This area is currently managed by Nezasa and will open to company users in a future release.
In the Workbench sidebar, click Skills and open the skill.
In the Skill Advisor panel, click Review skill for a critique of the skill's routing texts and content, or Improve skill for a proposed rewrite — optionally steered by Focus instructions. Apply what you like.
Track your runs
In the Workbench sidebar, click Advisor (under the AI Platform section).
Every evaluation, report-run analysis, and comparison is listed with its status and progress. Open a run to revisit its findings, or cancel or delete it from the row menu.
Tips and limits
Advisor runs use your company's AI usage allowance, like chats and reports — see Billing. Runs are capped per day.
An evaluation covers up to 10 conversations. For a representative picture, mix routine conversations with ones that went wrong.
Built-in agents can be evaluated but not modified — the Advisor still shows its findings and proposals, but agent changes can only be applied to company and personal agents. Use the findings to shape your own custom agents.
Focus instructions steer what the Advisor looks at; without them it applies the full quality rubric.
The Advisor proposes, you decide: no proposal is ever applied without your explicit confirmation, and prompt changes always show a diff first.
For help while writing an agent — drafting, reviewing, and improving a system prompt before it has any conversations — use the Prompt Author & Advisor in the agent editor. See Custom agents.
Related