The problem
The client operates a mid-size electronics manufacturing facility where a production line stoppage triggers a manual, multi-step investigation. An operator checks the machine's error logs, a maintenance technician cross-references them against a maintenance history spreadsheet, someone checks whether a similar fault happened recently on another line, and a shift supervisor decides whether to escalate to engineering. Done by people moving between systems that didn't talk to each other, this typically took 45 minutes to an hour before the root cause was identified, and the line stayed down the whole time.
A single agent wrapping one tool wasn't going to solve this. The investigation genuinely spans multiple distinct systems (machine telemetry, a maintenance ticketing system, a parts inventory database, and historical fault records) each with different access patterns and different domain logic for interpreting the data correctly.
Why this needed multiple specialized agents, not one generalist
Our first design used a single agent with access to all four systems as tools. It worked in simple cases but performed poorly on anything requiring genuine cross-system reasoning. A single prompt trying to hold expertise in telemetry interpretation, maintenance history, and parts availability at once produced shallow reasoning in all three areas rather than depth in any.
We restructured around four specialist agents, each with a narrow domain and its own tool access, coordinated by an orchestrator agent that decomposes an incoming fault report into sub-questions and routes them to the right specialist.
Architecture
The five agents
- Telemetry agent. Queries the machine's error logs and sensor history. It specializes in the facility's own fault-code taxonomy, which isn't standardized and took real input from the client's engineers to encode correctly.
- Maintenance history agent. Searches structured and unstructured maintenance records for similar past faults on the same or comparable machines, using retrieval over historical ticket text alongside structured fields.
- Cross-line pattern agent. Checks whether the same fault signature has appeared on other production lines recently. That's often the signal that separates a one-off mechanical failure from a systemic issue worth escalating immediately.
- Parts and inventory agent. If a likely root cause implicates a specific component, checks real-time inventory for the replacement part and expected lead time if it's not in stock.
- Orchestrator. Decomposes the incoming fault report, dispatches to the relevant specialist agents (not always all four; a fault might not need the parts agent involved yet), and synthesizes their findings into a single triage recommendation for the shift supervisor.
Tool-use reliability as a first-class concern
With five agents calling into different backend systems, tool-use reliability (calling the right tool with well-formed arguments) was a central design and testing concern. Each agent's tool schemas use strict typing and validation, and a dedicated evaluation harness measures the correct tool-use rate separately from the correctness of the final recommendation, because a wrong tool call can silently produce a plausible-looking wrong answer.
Human-in-the-loop by design, not as a limitation
The system produces a triage recommendation, not an autonomous action. The shift supervisor sees the orchestrator's synthesized reasoning and each specialist agent's individual findings, and makes the final escalation decision. We designed it this way deliberately: a wrong root-cause guess acted on autonomously on a live production line is much worse than a slightly slower, human-reviewed recommendation, and the client's engineering leadership was clear that this was non-negotiable for a first version.
Challenges and tradeoffs
- Fault-code taxonomy encoding took real domain expert time. The facility's error codes weren't documented in a form the telemetry agent could use. We spent significant time with a senior maintenance engineer building a structured reference the agent could reliably interpret. It was slower than anticipated, but it produced most of that agent's accuracy gain.
- Orchestrator latency added up across five potential agent calls. A complex fault could involve all four specialists plus the orchestrator's synthesis step, adding real end-to-end latency. We ran independent specialist calls in parallel (telemetry and cross-line pattern checks don't depend on each other) instead of sequentially, which cut typical response time significantly.
- Measuring "correct root cause" required a labeled evaluation set we didn't have. We worked with the client's maintenance team to build a labeled set of past fault investigations with confirmed root causes. Without it, we'd have had no reliable way to tell whether the multi-agent redesign was actually more accurate or just more complex.
Results
With the system's recommendation as the shift supervisor's starting point, root-cause triage time dropped from a 45-to-60-minute manual process to a median of roughly 13 minutes, a 3.5-times improvement measured over the first two months in production. The correct tool-use rate across all five agents held at 92% in the production evaluation harness. Failures are monitored, and when confidence is low the system falls back to showing the supervisor raw system data instead of a synthesized recommendation.
The client's maintenance engineering lead noted that the cross-line pattern agent, almost cut from scope during planning as a "nice to have," caught two systemic issues in its first month that would likely have been treated as isolated incidents under the old manual process.
What we'd do differently
We'd build the labeled evaluation set of past fault investigations earlier, ideally during discovery rather than after the single-agent prototype had been tested informally. Having it from the start would have let us make the single-versus-multi-agent decision with real numbers from day one instead of largely qualitative early impressions.