Build a multi-agent triage system with a supervisor and small workers

A support queue is a useful place to test multi-agent routing. Tickets arrive in different shapes: a billing question, an account-access problem or a bug report. Sending every ticket through one large prompt makes the agent responsible for classification, policy, troubleshooting and response writing at once. That is harder to test and gives each request more context than it needs.

A small agent supervisor pattern separates those jobs. A top-level supervisor classifies the ticket and delegates it to one specialized sub-agent. The billing worker gets billing instructions. The access worker gets account-recovery guidance. The bug worker gets a reproduction checklist. Your application owns this hierarchy; ZES provides model access through one API, with routing that lets you switch providers without code changes.

Draw the boundary before choosing models

Start by listing the domains the system can handle and the actions each worker may take. For a first version, let workers draft answers or recommend next steps, but do not give them unrestricted account changes or refunds. Keep authorization and any consequential action in ordinary application code, with a human review step where your policy requires one.

This boundary matters more than how many agents you create. A worker is a separately configured call with a narrow instruction set and tool access, not an independent service that can do anything the model suggests. For a broader discussion of where those boundaries belong, see why agent architecture debates turn on boundaries, not autonomy.

Make routing a structured decision

Have the supervisor return a small, validated object rather than a free-form explanation. For example, its result can contain a category drawn from billing, access, bug or unknown, a confidence value, and a short summary. The application checks that the category is allowed, then calls the matching worker. If the result is malformed or uncertain, send the ticket to a fallback path instead of guessing.

In language-neutral pseudocode, the control flow looks like this:

ticket = validate_incoming_request()

route = call_supervisor(ticket_summary)

if route.category is allowed and route.confidence meets your threshold: result = call_worker(route.category, minimal_ticket_fields)

else: result = send_to_review_queue(ticket)

validated = check_worker_output(result)

return draft_or_escalate(validated)

The point is not the syntax. It is the contract. Define the supervisor's allowed categories, the fields passed to workers, and the output each worker must return. Validate every boundary in code. A worker response might include a draft, the facts it relied on and a status such as answered, needs_more_information or escalate. Reject unknown status values and do not treat a confident tone as evidence.

Keep specialist context small

Give a worker only the information it needs. A billing worker may need the customer's question and relevant billing policy, not the entire conversation history or internal bug-reporting instructions. A bug worker can receive the error description and environment details without seeing unrelated account data. This reduces unnecessary context and makes it easier to spot which instruction or tool shaped an answer.

Keep worker toolkits similarly narrow. If a worker can retrieve policy, expose only the policy lookup it needs. If a task needs a write action, place that action behind application checks rather than allowing a model response to execute it directly. The supervisor should route; it should not quietly inherit every specialist's permissions.

Connect model calls through ZES

Use ZES for inference calls in the supervisor and workers. Its single API provides access to models from Meta, Mistral, Anthropic, OpenAI, Google and Cohere. Its model routing lets teams switch providers without changing application code. That gives engineers room to test different model choices for classification and specialist work, but it does not decide the agent workflow for you: delegation, schemas, validation and retries remain application responsibilities.

Start with one model choice per role and measure before adding complexity. A short classification task and a policy-heavy answer may have different requirements. Compare routing accuracy, output validity, latency and per-token usage on a representative set of tickets. ZES offers a pay-as-you-go Developer plan with per-token pricing; actual spend depends on usage. Its API playground can help inspect requests, and its real-time analytics dashboard can help monitor inference metrics, cost and performance.

Test the failure paths, not just the happy path

Build a test set with ambiguous requests, mixed-domain tickets, missing details, malformed worker output and cases outside the supported categories. Check that the supervisor selects the fallback for uncertainty, that workers return the required structure, and that retries do not duplicate a consequential action. Track how often tickets are misrouted and how often a worker escalates. These are more useful early signals than a polished demo.

There are trade-offs. A supervisor adds a model call before the worker call, so it can increase latency and token use. More workers mean more prompts and more cases to maintain. A narrow specialist can still give a wrong answer, and structured output makes a result easier to validate, not automatically correct. Begin with the fewest domains that reflect real queue volume, then split a worker only when tests show a distinct policy or tool boundary.

ZES says its inference can auto-scale from zero to millions of requests and offers edge deployments across more than 200 locations with sub-100ms latency. Those platform properties do not guarantee that a multi-step workflow meets its own response-time target. Measure the complete path, including supervisor, worker, validation and any human handoff. The useful architecture is the one whose routing decisions and failure behavior your team can explain and audit.