If your AI keeps giving the wrong return policy, start by checking what you gave it. A better prompt cannot fix three different policies that all look current.
An AI customer service knowledge base is the information your agent uses to answer questions. For an ecommerce store, that usually means product facts, shipping coverage, return terms and the instructions your team follows when something goes wrong.
You do not need a huge library to get started. You need clear answers to the questions people actually ask, plus a way to keep those answers current.
Start with your inbox
Read a sample of recent customer conversations. Pull out the repeated questions and look at the replies that actually resolved them. Keep the useful facts. Remove customer names, order numbers and other personal information before using examples in a general knowledge source.
“Do you ship to Canada?” is a knowledge question. “Where is my order?” also needs current order information. Keep that distinction clear. A policy document can explain shipping coverage. It cannot tell you where a particular package is today.
- Product questions: Materials, sizes, compatibility, care and the differences between similar items.
- Shipping questions: Destinations, processing windows and where customers can find current delivery information.
- Returns: Eligibility, time limits, exceptions and how a customer starts a request.
- Problems: Missing items, damaged products and situations that need your team.
Use those groups to decide what to prepare first. There is little value in writing 40 rare answers while the same shipping question keeps coming back unanswered.
Give each policy one clear source
A customer should not get one return window from your website and another from your support agent. Find the policy your business actually follows, make it clear and remove conflicting versions from the material the agent uses.
For each topic, record the approved answer, its exceptions, the source and the person responsible for keeping it current. Add a review date. You can use our customer service knowledge worksheet to start. It is a CSV you can open in your spreadsheet app.
For example, a return entry might say: unused items can be requested for return within 30 days of delivery, with the listed exclusions. That is a made up policy, not a recommendation for your store. The important part is that the trigger date, condition and exceptions are explicit.
Do not leave the agent to guess whether “30 days” starts at purchase, dispatch or delivery. Do not bury an exception in a document that contradicts the answer everywhere else.
Separate facts from instructions about tone
“Keep it friendly” is a tone instruction. “Do not promise a delivery date without current tracking information” is an answer rule. Your agent needs both, and they should be easy to tell apart.
A short voice instruction might be: “Answer the question first. Keep replies brief. Ask one question at a time. Skip the long introduction.” Then give a few examples of wording your team would actually send.
If the policy confirms that you ship to Canada, the answer can be: “Yes, we ship to Canada.” You do not need a paragraph about your commitment to serving international customers.
Preparing knowledge and examples is often called training a chatbot. It does not necessarily mean training a new model. In SmartReply, start with the brand knowledge and reply instructions available in your setup. Keep product and order connections responsible for information that changes.
Write down when the agent should stop
A useful knowledge base includes what the agent should not decide. If a damaged item needs a photo and review, say so. If a refund exception needs approval, identify the next step instead of asking the AI to “use its judgment.”
- The answer cannot be found in the approved information.
- Two sources disagree about the policy.
- The customer wants an exception or an action outside the agent's permissions.
- The order information does not match the customer's account or request.
Give each case somewhere to go. An instruction to escalate is incomplete if nobody knows who receives the conversation or what happens next.
Test the knowledge before customers depend on it
Build a small test set from your common questions and the mistakes you want to avoid. Include easy questions, messy wording and requests the agent should not answer on its own.
- Ask a question with a clear answer in the source.
- Ask the same question in different words.
- Ask about an exception that needs a person.
- Ask for a product fact that is not in the source.
- Check the answer after correcting an outdated policy.
Use the AI support test worksheet to record expected and actual answers. Read the actual response. Check the policy, the next step and the amount of information requested. A fluent answer can still be wrong. Keep the failed questions and run them again after changes.
If you use SmartReply with an assistant, our ChatGPT customer service workflow explains how to inspect instructions and request a sandbox test. Check a completed test before changing live behavior.
Keep the knowledge attached to someone who owns it
Policies change. Products sell out. Holiday shipping windows end. Decide who updates the source and who checks the resulting replies. Review after a policy or catalog change, and whenever customers keep correcting the same answer.
When an agent gets something wrong, check whether it lacked the fact, found the wrong source or failed to follow an instruction. Those are different problems. Fixing the source is usually more useful than adding another sentence saying “be accurate.”
Start with the topics causing the most repeat work. Add the approved information to your SmartReply support setup, test it and review a sample of conversations after enabling it. If you serve customers in several languages, the next step is checking those answers across languages.