How to Test Intercom Fin Before Launch

Before opening Batch Test, write down what Fin is allowed to answer on day one. Keep the first boundary narrow: common how-to questions, low-risk billing explanations, simple account navigation, and topics with one current source of truth.

Then list what Fin should not answer yet. This usually includes refunds with exceptions, contract changes, access recovery, security requests, legal language, reseller billing, high-value accounts, and anything where agents still use private judgement.

Use recent conversations and topic groups first because they reflect the language customers actually use. Then add a CSV group for the edge cases that may not appear often but define launch risk: cancellation timing, billing exceptions, admin access, data deletion, and escalation requests.

Intercom supports question generation from past conversations, topic-based question groups, manual questions, and CSV upload. A strong pre-launch run uses more than one method: real conversation phrasing for coverage, and curated CSV questions for risk.

A Fin answer should not pass because it sounds reasonable. It passes when it used the right help article, snippet, guidance, procedure, data connector, custom answer, or automation path for the customer intent.

Record source failures separately from writing failures. If Fin used the wrong article, the fix is source routing or content cleanup. If it used the right source but skipped a condition, the fix may be guidance, procedure logic, or a clearer article.

Do not collapse the result into good or bad. The launch decision needs more nuance. An answer can be accurate but still restricted because it depends on plan, audience, region, account status, or a human approval step.

Approve only when the answer is current, complete, cited, and safe for the chosen audience. Restrict when Fin should ask a clarifying question or hand off under specific conditions. Block when the source is missing, conflicting, stale, or too risky for automation.

The first test run is only useful if the team re-runs it after fixes. Keep stable Batch Test groups for top intents, high-risk intents, and newly changed policies. When a help article, snippet, guidance rule, procedure, connector, or automation changes, rerun the affected group before expanding Fin's scope.

Meihaku turns the retest into an approved-intent map. The support team can see which intents moved from blocked to approved, which remain restricted, and which should stay out of automation until the underlying source is defensible.