AI Support Hallucination Examples: What to Test Before Launch

The Air Canada chatbot case is the cleanest warning for support leaders. A customer relied on a chatbot answer about bereavement fare timing, but the answer contradicted the airline's actual policy. The British Columbia Civil Resolution Tribunal held Air Canada responsible for the information on its website, including the chatbot.

The DPD incident showed a different failure mode. The customer could not get useful parcel help, then prompted the chatbot into jokes, profanity, and criticism of the company. DPD said an error followed a system update and disabled the AI element while updating it.

The Cursor incident showed how a bot can invent a plausible company rule. Users who were being logged out across devices received support email saying the behavior came from a one-device policy. Cursor later clarified that no such policy existed and that the answer was wrong.

Different incidents, same support problem: the AI answered outside a defensible boundary. It did not have enough source control, escalation control, or uncertainty handling to protect the customer and the company at the same time.

Policy invention is the most dangerous support hallucination because it sounds like an official company decision. The customer does not see a model filling a gap. They see the brand giving them a rule.

This usually happens when the AI receives a question that is close to existing policy but not fully covered by the source. It may merge two partial facts, infer an exception, or explain a bug as if it were intended behavior.

Before launch, every billing, refund, cancellation, access, security, and account-change answer should point to the exact source line that allows the claim. If the source does not contain the condition, the AI should ask for clarification or hand off.

The DPD case was not mainly about knowledge retrieval. It was about behavior control. The chatbot followed the customer's playful or adversarial prompts instead of staying inside a narrow support task.

Support teams should test prompt-injection and role-break attempts before launch, but those tests should be tied to real service context. The bot should remain helpful when it cannot locate an order, should offer a valid handoff path, and should not improvise jokes, opinions, or brand commentary.

A safe support AI does not need to win an argument with the customer. It needs to keep the conversation inside the approved support workflow, then escalate when it can no longer help.

Many AI support failures happen because the bot answers when it should say it does not know. In support, uncertainty is not a writing problem. It is a routing problem.

If a customer asks about eligibility, refund exceptions, security access, or product behavior that changed recently, the correct answer may depend on account state or an internal owner. The AI should not convert missing context into a universal rule.

The pre-launch test should reward honest uncertainty. A handoff can be a passing result when the source is missing, stale, or ambiguous. A smooth but unsupported answer should fail.

Start with recent tickets, chats, macros, and policy docs. Group them into customer intents, then write tests from the customer's actual language. Keep typos, frustration, partial context, and multi-intent requests.

For each test, attach the canonical source before reviewing the AI answer. Reviewers should be able to say whether the answer is supported, missing a condition, using the wrong source, or answering a topic that should be human-only.

Do not measure only pass rate. A test set full of easy FAQ questions can hide the one refund, cancellation, security, or legal answer that will damage trust. Weight the risky intents more heavily than the obvious ones.

Pre-launch testing reduces risk, but it does not end the QA loop. Policies change, vendors update models, help-center articles drift, and customers ask new combinations of old questions.

After launch, track wrong-answer rate, human override rate, unresolved escalation, re-contact within 48 or 72 hours, AI-only CSAT, and newly blocked intents. These signals reveal whether the AI resolved the case or merely moved the cost somewhere else.

The weekly operating question should be simple: what did the AI answer this week that we would not approve again? That answer becomes the source-fix, prompt-fix, vendor-config, or human-only backlog.