The problem
The client is a regional telecom provider offering broadband and mobile service. Its tier-1 support queue was large and, when we analyzed three months of ticket data, surprisingly concentrated: billing questions, outage status checks, and plan-change requests together made up well over half of all tier-1 volume. Yet they were routed through the same general queue as genuinely complex technical issues, so simple questions waited behind complicated ones, and agents spent real time on repetitive requests instead of the harder cases that needed their judgment.
The client had looked at broader AI chatbot options and been wary of what they saw: general-purpose support bots that tried to answer anything and, as a result, answered plenty of things incorrectly, including a competitor's publicized incident where a bot gave a customer wrong information about a contract cancellation policy. Their ask was narrower and, we thought, smarter: don't try to cover everything; cover the few categories that make up most of the volume, and do those well.
What we actually did
We built a chatbot scoped to exactly three ticket categories (billing questions, outage status, and plan changes), with a hard boundary that routes anything outside those categories to a human agent immediately rather than attempting a best-effort answer.
Weeks 1-2: scoping the three categories down to what's actually answerable safely
Within even these three categories, not everything was safe for a bot to handle autonomously. Billing questions split into "what's my current balance and when is it due?" (safe, and answerable from account data) and "why was I charged this specific amount?" (often involving a promotional credit or proration that the bot could explain but shouldn't resolve unilaterally). We drew this line carefully with the client's billing team, because getting it wrong in either direction (over-promising a refund, or deflecting something it should have handled) had real cost.
Weeks 3-4: grounding answers in live account and network data, not static FAQ content
Outage status had to be grounded in the client's live network-status system, not a static "known issues" page that could be stale during an actual outage. A bot confidently telling a customer there are no known outages in their area during an active outage would be worse than having no bot at all. Billing and plan-change responses are grounded in the customer's actual account data, retrieved through the client's existing customer-account API with read access for billing lookups and a scoped, confirmation-gated write path for plan changes.
Week 5: the hard refusal boundary and human handoff
Any question outside the three scoped categories, or any in-scope question where the bot's confidence falls below a defined threshold, goes immediately to a human agent with the conversation context carried over. We set this threshold deliberately conservative at launch, preferring that the bot hand off a question it probably could have answered rather than risk answering one it shouldn't.
Challenges and tradeoffs
- Drawing the line within the billing category required real negotiation with the client's billing team, not just an engineering decision. The distinction between "explain a charge" and "resolve a billing dispute" wasn't obvious from ticket transcripts alone, and the billing team had legitimate concerns about the bot creating de facto refund policy through how it explained charges. We iterated on the bot's exact phrasing for charge explanations under their review before launch, to avoid language that sounded like a promise the bot couldn't back up.
- The live network-status integration surfaced data-quality issues in the client's own outage-reporting system. Building the outage-status grounding exposed gaps in the client's internal outage system: some genuine outages weren't being flagged promptly by field technicians. We couldn't fix that ourselves, but we flagged it clearly to the network operations team as a dependency the bot's accuracy relied on, rather than quietly absorbing the resulting inaccuracy as our own bug.
- Being conservative with the confidence threshold at launch meant a higher initial handoff rate than the client's ROI projections had assumed. We were upfront that a cautious launch threshold would mean lower deflection in the first weeks than a looser one, trading some early ROI for a lower risk of a bad answer damaging trust before the tool had a track record. The client's support director agreed to this tradeoff explicitly; we didn't make the call unilaterally.
Results
In the first full quarter live, 58% of tier-1 tickets were fully deflected (resolved by the bot with no human agent involved), concentrated in the three scoped categories; technical troubleshooting and everything else continued to route to human agents as designed. Containment accuracy, the share of deflected conversations with no follow-up ticket on the same issue within 48 hours, held at 91% across the quarter. The client's support director considers that more important than the raw deflection rate, because high deflection with poor containment would just delay tickets rather than resolve them.
In the client's internal quarterly survey, support agents reported that their remaining queue is meaningfully more focused on issues that need real judgment. The support director attributes this to the narrow, disciplined scoping rather than to the bot's underlying model.
What we'd do differently
We'd raise the outage-reporting data-quality question with the client's network operations team in week one, instead of in week three when we first hit it while building the live-status grounding. It didn't block the launch, but earlier visibility would have given their team more runway to address it before the bot's accuracy in that category depended on a known gap in their own systems.