AI customer service sounds simple. It’s not.

I’ve built it three times. Cost me about $15,000 in total. Here’s what actually works.

Attempt 1: ChatGPT API ($3,000)

The plan: Feed ChatGPT our FAQ. Let it answer questions. Simple.

The build: Two weeks. Basic API integration. Prompt with FAQ content.

The result: Disaster.

Hallucinations everywhere. Made up return policies. Invented features we didn’t have. One customer got told they could return a product after 90 days. Our policy is 30 days.

Lesson: Raw LLMs lie confidently. You can’t trust them with customer-facing information.

Attempt 2: Fine-tuned Model ($8,000)

The plan: Fine-tune on our support history. Train it to respond like our team.

The build: Six weeks. Hired a consultant. Prepared training data. Fine-tuned GPT-3.5.

The result: Better, but not good enough.

Fewer hallucinations but still present. And it learned our bad habits too. The support team sometimes gave inconsistent answers. The model reproduced that inconsistency.

Cost: $4,000 in consulting plus $4,000 in development time.

Lesson: Fine-tuning is expensive and doesn’t solve the fundamental problem.

Attempt 3: RAG with Guardrails ($4,000)

The plan: Don’t trust the model. Retrieve verified information. Constrain responses.

The architecture:

1. User asks question
2. System searches knowledge base for relevant docs
3. Model answers using ONLY retrieved context
4. Confidence check before sending
5. Low confidence routes to humans

The build: Three weeks. Used existing frameworks. Focus on guardrails.

The result: Works.

85% of Tier 1 questions handled automatically. 15% routed to humans. Zero hallucinations in three months.

The Working Architecture

Knowledge base: Notion database with verified answers. Each answer has sources and last-verified date.

Retrieval: Vector search using OpenAI embeddings. Finds relevant answers from knowledge base.

Generation: Claude API. Instructed to ONLY use retrieved context. Never make up information.

Guardrails:

• If no relevant context found, route to human
• If answer requires decision-making, route to human
• If sentiment is angry, route to human
• If topic is sensitive (refunds, complaints), route to human

Human handoff: Seamless transition. Customer doesn’t know AI was involved.

The Cost Breakdown

Development: 60 hours at $50/hour = $3,000 Infrastructure: $200/month (embedding storage, API calls) Knowledge base maintenance: 5 hours/month

First year total: ~$5,400

Savings: Reduced support tickets by 40%. At our volume, that’s $20,000/year in support costs.

ROI: Clear win.

What I’d Do Differently

Start with the knowledge base

Don’t touch AI until you have clean, verified, organized documentation. Garbage in, garbage out.

We spent a week just organizing our FAQ and support processes. That week was the most valuable investment.

Involve support team early

They know the edge cases. They know what questions are easy vs hard. They know what shouldn’t be automated.

Our support lead reviewed every automated response type. Her input caught problems we’d have missed.

Plan for human handoff

The goal isn’t 100% automation. The goal is 80% automation with 100% quality.

Design the human handoff first. Make it seamless. Then automate what’s safe.

When To Build This

Build AI customer service when:

• You have 500+ support tickets/month
• Most tickets are Tier 1 (simple, repetitive)
• You have documented answers to common questions
• You can tolerate a 3-month implementation

Don’t build when:

• Low ticket volume (just answer them manually)
• Mostly complex issues requiring judgment
• No documented knowledge base
• You need it working tomorrow

If you’re ready but don’t have internal expertise, find AI consultants Sydney. The implementation details matter more than the concept.

The Bottom Line

AI customer service works. But only with constraints.

Don’t trust the AI to know things. Make it look things up.

Don’t let it make decisions. Route those to humans.

Don’t expect magic. Expect augmentation.

Get those right, and you’ll actually help customers while saving money. Get them wrong, and you’ll create problems that cost more than they save.