How We Built a Customer Support Agent That Handles 80% of Tickets Without Human Help

Last year a client came to us with a support problem. They were running a mid-sized SaaS product, about 1,400 customers, and their support queue had become a full-time job for two people. Tickets were piling up over the weekend, response times were slipping, and customers were noticing.

They had looked at Intercom, Zendesk, Freshdesk. All of them had "AI features" attached. None of them solved the actual problem, which was: their product is complex, their documentation is dense, and templated chatbot replies were making customers more frustrated, not less.

So we built something custom. It has now been running for six months. It resolves about 80% of tickets on its own. Here is what it actually took.

The Problem With Off-the-Shelf Support Bots

Most support chatbots work the same way. They match keywords, pull canned answers from a FAQ, and escalate when they do not recognize the query. That works fine if your product is simple and your customers ask the same five questions.

This client's customers did not. They were asking questions like "why is my webhook payload missing the created_at field when I use batch mode?" Not exactly something a keyword matcher handles well.

The difference between a chatbot and what we built is that our agent actually reads the docs, reads the support history, and reasons about the specific question being asked. It generates a fresh answer each time, not a retrieval hit.

What We Actually Built

The system has three pieces.

First, a knowledge base. We took every piece of documentation, every changelog entry, every Notion page the client had and indexed it into a vector store. We set up a pipeline that re-indexes every night, so when the product ships a new feature, the agent knows about it the next morning.

Second, a support history layer. We pulled in two years of resolved tickets, tagged by the final solution. When a new ticket comes in, the agent searches this history first. If a similar ticket was solved before, it uses that as a starting point. This alone cut hallucinations significantly, because the agent is grounded in what has actually worked.

Third, an escalation trigger. If the agent's confidence score drops below a threshold, or if the customer uses certain phrases ("this is urgent", "billing issue", "thinking about canceling"), it routes to a human immediately, with a summary of what it already tried. The human does not start from scratch.

The Part Nobody Talks About: Training the Escalation

Getting the 80% right was not the hard part. Getting the other 20% to escalate gracefully was.

Early versions of the agent would try to answer everything. It was technically confident when it should not have been. A customer asking about an edge case in the billing system would get a plausible-sounding but wrong answer, which is worse than no answer.

We spent about three weeks tuning the escalation logic. We reviewed every case where the agent answered but should have escalated, reverse-engineered the patterns, and added explicit rules. Things like: any ticket mentioning a specific dollar amount escalates. Any ticket about data loss escalates. Any ticket where the customer has opened more than 3 tickets in the last 30 days escalates to a senior rep.

The rule-based layer sitting on top of the AI layer is what makes it trustworthy. The AI handles the reasoning, the rules handle the liability.

The Integration Work

The client was already using Zendesk. We did not replace it. The agent plugs in via Zendesk's API, reads incoming tickets, and either posts a reply and closes the ticket, or adds an internal note and assigns to a human.

This took about two weeks to build cleanly. Zendesk's API is well-documented but has some quirks around ticket status transitions. We also had to build a small admin panel so the support team could review what the agent was doing, flag bad answers, and add new rules without touching code.

That admin panel was not in the original scope. It took an extra week. But it is what gives the support team ownership over the system, which is why they actually trust it.

What the Numbers Look Like Now

Six months in, the agent handles 80% of tickets with no human involvement. Average first-response time went from 6 hours to under 3 minutes. The two support staff now spend their time on the hard cases, and they report being less burned out because the queue is not a constant flood.

Customer satisfaction scores went up 11 points in the first quarter. Not because the AI is warmer than a human, but because customers get answers immediately instead of waiting overnight.

The system costs about $400 a month to run in API and infrastructure costs. The two support staff freed up roughly 60% of their time. The math is not close.

What This Takes to Build

A project like this is not a weekend build, but it is also not a six-month engagement. We scoped it at 6 weeks. The real investment is in understanding the product deeply enough to know what the agent should and should not answer.

If you have a support queue that is growing faster than your team can handle, and your product is complex enough that templated bots have already failed you, this is a solvable problem. We have built this pattern enough times now that the core scaffolding is fast. The work is in the tuning.

If you want to talk through whether something like this fits your situation, we are straightforward about whether it makes sense. Reach out to the SimplyShip App team and we will tell you honestly.