Yes, and you can try it today. Put a few documents, such as your FAQs, price list and return policy, into a project or custom GPT in ChatGPT, a project in Claude, or a Gem in Gemini. Then write one instruction: "Answer only from these documents, say which one you used, and if the answer isn't there, say a colleague needs to confirm." When a customer asks something, your team pastes the question in and gets a draft reply based on your documents, which they check before replying.

At many companies, customer service runs on experienced staff and a handful of policy documents, and the answers are accurate and friendly. When AI comes up, the first question is usually "Will AI just make things up?" That's fair: AI sounds fluent, but it doesn't know your return policy, latest prices or this week's shipping schedule, so it may invent a plausible answer. This is called hallucination. Having AI look in your data first and answer from what it found is the basic idea behind RAG (retrieval-augmented generation). This article starts with prompts you can copy, then covers what to check, how to organize your documents, and the next step when you want customers to ask directly.

Ask AI directly: 3 prompts to copy

Swap in your own company, products and policies (the ones below are examples).

Instructions for the project (paste into the instructions field of your project, GPT or Gem)

You are the customer service assistant for [company name], drafting replies for our team. Answer only from the attached documents: our FAQs, price list and return policy. With every answer, say which document and which section it's based on. If the answer isn't in the documents, or the question involves refund amounts, contracts, or medical or legal matters, reply: "I'll need to check with a colleague on this. Could you leave your contact details?" Don't guess. Keep the tone friendly and short, suited to a reply on LINE.

Draft a reply

A customer asks: "I bought the Chamomile Soothing Lotion last week and I've opened it. Can I still return it? Who pays for shipping?" Draft a reply based on the documents, and say which section it's based on. If the documents don't say whether opened items can be returned, follow the instructions and pass it to a colleague.

Check with test questions

Attached are 20 questions real customers have asked, with the correct answers. Answer each one using only the documents in this project, then compare with the correct answer. List the questions it got wrong, answered incompletely or couldn't find a source for, and say which document may be missing something or unclear. Give me the result as a table.

After AI drafts a reply, check these 3 things

  1. Does the source really say that? Open the document and section it cites and make sure it actually says this. Don't use an answer that can't point to a source.
  2. Is it quoting an old version? When two documents cover the same topic, it may quote the old one. Keep only the latest version in the project, and swap the file as soon as a policy changes.
  3. Did it hand off what it should? For refund amounts, contracts and complaints, check that it used the standard reply and passed it to a colleague.

Only put information you could share with customers in the project. Before pasting in a customer's question, delete their name, phone number and order number, or use your company's enterprise plan.

Make it easier: your documents and AI working together

1. List your common questions and standard answers

Go through past LINE chats (LINE is the messaging app most people in Taiwan use), emails and call notes, and pick the 10–20 questions customers ask most. In an Excel or Google Sheets file, add columns for the question, standard answer, source document, last updated date and owner. Put it in the project, and the most common questions have a ready answer to quote.

2. Keep one version of each document

With several versions of the same price list or return policy, neither people nor AI can easily tell which one is right. Move old versions into an "Archive" folder, put "Last updated: [date]" at the top of each document, and from then on edit only that one.

3. Use headings, and write in your customers' words

Use heading styles in Word or Google Docs, such as "Return deadline" and "Return shipping," with one topic per section, and avoid "as mentioned above." If a customer asks "Can I return this?" but the document says "RMA process," the two won't easily match, so add the words customers actually use. Write out key points from tables and images as text too.

4. Build a test set

Pick 50–100 real questions from past customer service logs, and write down the correct answer and source document for each. Use it to train new staff and to check AI with the third prompt above; later, use it to sign off a full AI customer service setup.

How does RAG work?

A project is a good way to try this out, with your team using it. When you want customers on your website or LINE to ask directly, and the documents keep growing, RAG is the usual approach. It has two phases:

Preparing in advance: your documents are split along their headings into chunks (a few hundred words each, say), each covering one complete topic. An embedding model turns each chunk into a vector that represents its meaning, stored in a vector database.

When someone asks a question:

  1. Retrieve: find the few chunks closest to the question in meaning, usually combined with keyword search so exact terms like model and part numbers aren't missed.
  2. Assemble: hand the retrieved chunks to the language model along with the question, with explicit instructions: "Answer only from the material below. If the answer isn't there, say you don't know."
  3. Answer with sources: the AI writes its answer and notes which documents it drew on.
The RAG pipeline: documents are chunked and indexed; when a question comes in, the relevant chunks are retrieved first and passed to the model, which writes an answer with sources
The two phases of RAG: chunk and index your documents in advance; for every question, retrieve first, then answer, with sources.

When to consider the next step

A project plus well-organized documents can serve you for a long time. It's worth looking at the next step when:

  • Your team copies customer questions into AI and pastes the replies back into LINE or email every day.
  • You'd like customers on your website or LINE to ask directly and get an answer with a source, after hours too.
  • Your documents change often, and you'd like AI to always work from the latest version without swapping files.
  • You need every conversation and the material it drew on logged, to see why an answer went wrong.

The next step: AI customer service that looks up your documents before answering

Your documents stay where they are. When they're updated, the index is rebuilt, so AI finds the latest version. When a customer asks on your website or LINE, AI looks it up first, then answers, and every conversation is kept in the admin panel.

When it can't find an answer, a person takes over

A trustworthy AI customer service assistant has to know what it doesn't know. These are rules we always build in, and most come straight from the instructions you wrote earlier:

  1. No relevant material, no detailed answer: use the standard reply, and ask for contact details or hand off to a person.
  2. Anything involving money, contracts, or medical or legal matters always goes to a person.
  3. Cite a source with every answer: customers can open the original text, and support staff can verify the answer quickly.
  4. Point customers to the right page: link to the "Returns & exchanges" page so customers can read the full policy themselves.
AI customer service answers that cite their sources; when no relevant data is found, it hands the conversation to a person instead
Good AI customer service cites its sources and points customers to the right page; when the answer isn't in the data, a person takes over.

How a rollout runs, and what to watch after launch

  1. Week 1: Organize the data. If you've done the steps above, this goes much faster.
  2. Week 2: Build the index and a test set. Use the test set you've already prepared.
  3. Week 3: Tune and sign off. Check each answer: is it correct, does it cite the right source, and does it hand off properly when it can't find the answer?
  4. Every week after launch: review the logs and add data. For questions it couldn't answer, add a document or an FAQ entry; use the chats handed to a person to find questions you can standardize. Then rerun the test set to make sure the other answers still hold.

We suggest a soft launch on your website first, with every conversation kept in the admin panel. Once it's stable, connect other channels, such as customer service on your LINE Official Account.

RAG isn't mysterious technology. At heart it's "look it up before you answer," just like a conscientious new support hire. To see where to start with your own data, try a free AI Health Check, or look at our AI services and website design and development. You're also welcome to come talk to us.