AI customer support promises cost savings and 24/7 availability. Reality is messier.

We implemented AI support 8 months ago. Here’s what actually happened.

The Setup

Before AI:

• 2 full-time support agents
• Average response time: 4 hours
• CSAT: 4.2/5
• Cost: ~$120K/year in salary

After AI (current state):

• 1 full-time support agent + AI
• Average response time: 2 minutes (AI) + 2 hours (human escalation)
• CSAT: 4.0/5
• Cost: ~$70K/year salary + $8K/year AI tools

Savings are real. But so are the tradeoffs.

What We Built

Layer 1: AI Deflection

AI handles first contact. Uses our knowledge base to answer common questions.

Tools: Intercom Fin + custom fine-tuning Resolution rate: 40% of tickets fully resolved by AI Cost: ~$500/month

Layer 2: AI Triage

When AI can’t resolve, it categorizes and summarizes for human agents.

Impact: Humans handle complex issues faster. Context is already gathered. Time saved: ~15 minutes per escalated ticket

Layer 3: Human Agent

Complex issues, angry customers, edge cases.

Change from before: Fewer tickets, more complex on average. Required hiring for different skills.

What Worked

Response Time Improvement

Instant responses for common questions. Customers love it when the answer is immediate.

Before: 4 hours average After: 2 minutes for AI-resolved, 2.5 hours for escalated

For simple questions, this is a massive improvement.

After-Hours Coverage

AI handles tickets at 3 AM. Before, customers waited until morning.

Night and weekend resolution rate: 25% (lower than daytime, but better than 0%)

Agent Satisfaction

Support agents handle fewer repetitive tickets. They focus on interesting problems.

Agent turnover: Down. More meaningful work.

Knowledge Base Improvement

AI exposed gaps in our documentation. We fixed them. Documentation is now better for everyone.

What Didn’t Work

Initial Customer Frustration

Early version was too aggressive. Pushed AI responses when customers wanted humans.

Result: Customer complaints. “Stop talking to your robot.”

Fix: Clear “talk to human” option. AI discloses it’s AI. Never pretend.

Complex Issue Mishandling

AI confidently gave wrong answers for edge cases. Customers followed bad advice.

Result: Some angry customers. Some churn.

Fix: AI expresses uncertainty. Escalates when confidence is low. Never guesses on account-specific issues.

Training Overhead

AI needs constant tuning. Product changes require knowledge base updates. New edge cases require new training.

Estimated time: 5-10 hours/week maintaining AI support system.

Not set-and-forget. Ongoing work.

CSAT Dip

CSAT dropped from 4.2 to 4.0. Mostly from customers who wanted humans and got AI.

Acceptable tradeoff for us. May not be for everyone.

The Lessons

Lesson 1: Transparency Wins

Customers accept AI when you’re honest about it. They hate being tricked.

Always disclose: “I’m an AI assistant. I can connect you to a human anytime.”

Lesson 2: Easy Escalation

One click to reach a human. No friction. No “are you sure?”

Some people want humans. Let them have humans.

Lesson 3: Don’t Over-Automate

AI handles 40% of tickets. Tempting to push for 60% or 80%.

Don’t. The remaining tickets are complex for a reason. Forcing AI on them hurts customers.

Lesson 4: Monitor Quality

AI makes mistakes. You need to catch them.

We review 10% of AI responses randomly. Check for accuracy and tone.

Lesson 5: Train Humans Differently

Agents now handle only complex issues. They need different skills.

Before: Speed and efficiency After: Problem-solving and empathy

If you want help implementing AI support that actually works, consider working with AI consultants Brisbane. We made mistakes that experience would have prevented.

The ROI Calculation

Savings:

• $50K/year in staffing
• Improved response time
• 24/7 coverage

Costs:

• $8K/year in AI tools
• ~$30K/year equivalent in maintenance time
• CSAT risk

Net benefit: ~$12K/year plus intangible improvements.

Not transformative. But positive. ROI is real if you do it right.

Should You Do It?

Yes if:

• You have 500+ support tickets/month
• Many tickets are repetitive and documentable
• You have capacity to maintain the system
• You’re okay with some customer experience risk

No if:

• Low ticket volume (doesn’t justify setup cost)
• Highly complex, custom support needs
• Customer experience is your main differentiator
• You don’t have bandwidth to tune and monitor

AI support isn’t magic. It’s a tool. Use it where it fits.