How to Test AI Search Visibility Without Fooling Yourself

Testing your own business in an AI search tool is easy. Testing it in a way that tells you something true is harder.

The trap is not that the tools are unreliable, though they are variable. The trap is that a single answer looks like evidence. It arrives written in confident prose, it mentions real businesses, and it feels like a verdict. Act on it directly and you can spend weeks fixing a website that was never the problem.

This is a method for getting something useful out of that test instead.

Three ways people fool themselves

The flattering prompt. Asking “Is [business name] the best therapist in Canterbury?” tells you almost nothing. You have handed the tool the answer and asked it to agree. A prospective client does not know your name yet. That is the whole problem you are testing.

The single screenshot. One answer on one afternoon becomes “we rank in ChatGPT” or “we are invisible in AI search”. Neither is true. Answers shift with wording, location, session and the week. A screenshot records a moment, not a position.

The unexamined source. This is the expensive one, and almost nobody checks it. A tool says something wrong about your business, and you assume your website is unclear. So you rewrite pages, add structured data, tighten your headings. The wrong answer carries on, because it was never reading your website.

Ask a real customer question

Start with the question someone would actually type when they do not know you exist. Combine a service, a place and one practical decision factor:

• “Private anxiety counselling in Canterbury with evening appointments.”
• “Accountant in Kent for a contractor who needs help with tax returns.”
• “Family solicitor near Canterbury offering a fixed-fee first consultation.”

Keep the set small. Five or six questions you reuse every time. A consistent prompt set is what makes the next test comparable to this one.

Then find out what the tool actually read

Here is the step that changes what you do next.

AI search tools do not all work the same way. Some fetch pages live at the moment you ask. Others answer largely from training data and from summaries of directories and listing sites. Both can produce a confident paragraph about your business. Only one of them has seen your website.

So before you diagnose anything, check which happened. Use a tracer fact: something that is true on your current website and true nowhere else.

Good tracer facts are specific, recent and only on your site:

• A price you published in the last few weeks.
• A service you added at your most recent site update.
• An opening time, location detail or staff name that changed recently.
• A figure that appears in your page copy but has never been in a directory listing.

If the tool repeats your tracer fact, it read your site. If it repeats an older version of your details, such as a price you no longer charge, a service you dropped or an address you moved out of, it did not. It is summarising something else, and that something else is where your problem lives.

A practical example. A practice republishes its website with a £35 first appointment. The old site never listed a price at all. A week later, one tool quotes the £35. That tool is reading the live site, and reading it well. A second tool describes a service the practice stopped offering two years ago. No amount of work on the website will change that second answer, because the website does not say it. A directory does.

Two wrong answers, two different jobs

Once you know what the tool read, the fix follows from it.

If the tool read your site and still got it wrong, the problem is on the page. Something is missing, buried, ambiguous, or stated only in an image or a PDF. This is website work: clearer service pages, plain headings, facts in body text, a stated scope of what you do and do not offer. Our guide to why a business may not appear in AI search covers most of this ground.

If the tool did not read your site, the problem is off it, and no code change will touch it. The work is opening your directory profiles, professional register entries and Google Business Profile and correcting them by hand. It is dull, it needs account logins rather than a developer, and it is usually the higher-value job of the two.

This distinction matters commercially. The instinct when an AI tool gets you wrong is to spend money on the website. Sometimes that is right. Often the website is already fine and the money should go somewhere far less exciting.

Write down the conditions

For each question, record: the date, the tool, the exact prompt, your rough location, whether live search was on, the answer, which businesses were named, which sources appeared, and whether your tracer fact showed up.

That last column is the one most test sheets are missing. Without it, you are guessing at causes.

Keeping the record also stops a half-remembered answer becoming the reason for a decision three months later.

Read the patterns, not the verdict

One absence is not a technical failure. One mention is not visibility solved. Look across the whole set for things that repeat:

• Do competitors appear with a sharper hook than yours, such as a named specialism rather than a general service?
• Do the same directories keep surfacing? Those are worth your attention.
• Is there a question none of your pages directly answers?
• Are you described only in your own words, with no reviews or third-party evidence anywhere?

Each of those is a piece of work with an owner. That is the real output of the exercise: a short, prioritised list, with the website items separated from the profile items.

What this test cannot tell you

It cannot tell you whether you will be mentioned tomorrow. These are single observations of systems that change without notice, and two people asking the same question in the same hour may see different answers.

More importantly, no work on a website or a listing can make an AI tool cite, mention or recommend a business. Anyone selling that is selling something they do not control.

What the work does do is worth having on its own terms. The same details that let a tool describe you correctly are the details a cautious client checks before they get in touch. Correcting a wrong service on a directory helps the person reading that directory, whether or not any assistant ever reads it.

A sensible schedule

Quarterly is usually enough for a local service business, with an extra check after a website launch, a service change, a move or a profile update. Use the same prompt set each time and compare against your last record.

Re-test four to six weeks after making corrections. Directory changes take time to propagate, and some tools carry old information for a while after the source has been fixed.

Want this done properly and written down?

An SEO Audit runs a defined prompt set across AI search tools, checks what each one actually read, and separates the findings into website work and profile work, with the prompts, the evidence and the limits of the test recorded so you can repeat it yourself.

Next step: Request an SEO Audit