How to check if ChatGPT recommends your brand: a 30-minute manual test

A step-by-step method to test whether ChatGPT, Gemini and Claude name your brand when buyers ask for options, and how to read the results.

By Farzad Golestan, Founder of AnswerSeat 8 min read
AI ANSWERWhat is the best tool for my business?01Brand A02Brand B03Brand C04Your brand?ANSWERSEAT.COM / BLOG

Short answer

Ask ChatGPT, Gemini and Claude the questions your buyers ask, in their words, with web search on and in a fresh chat. Run each question three times per assistant. Write down which brands are named, in what order, and which pages are cited. If your brand is missing from most answers, you have an AI visibility gap, and the cited pages show you where the assistants learn about your category.

Why is a manual test worth 30 minutes?

More buyers now ask an AI assistant for a shortlist before they ever open a search results page. "Best invoicing app for freelancers" or "a cheaper email marketing tool for a small shop" gets a direct answer with three to five names. If yours is not one of them, that buyer may never reach your website.

Your analytics will not show you this. Google Search Console counts clicks from AI Overviews inside its normal web report, but it does not show what the answer said or which brands it named (Google documentation). ChatGPT, Gemini and Claude show you nothing at all. The only way to know is to ask them yourself, the way a buyer would.

Step 1: Which questions should you test?

Write ten questions a real buyer types before they know your name. Do not include your brand. Mix four kinds:

  • Category shortlists: "What is the best [category] for [audience]?"
  • Competitor alternatives: "What are good alternatives to [biggest competitor]?"
  • Constraints: "[category] for [use case] under [price] a month"
  • Problems: "How do I [job your product does] without [common pain]?"

Take the wording from places buyers actually talk: sales calls, support tickets, review sites, community forums in your niche. A question in your marketing language ("AI-powered revenue operations platform") tests a query nobody asks.

Example for an invoicing app: "What is the best invoicing app for freelancers in Germany?", "[Biggest competitor] alternatives for a one-person business", "How do I send invoices that get paid faster?"

Step 2: How do you avoid a biased answer?

Assistants adapt to the person asking. Saved memories, past chats and your location can all change which brands appear. To see roughly what a new buyer sees:

  • Use a temporary or incognito chat, or a fresh account with memory switched off.
  • Turn web search on, because most buying questions trigger a live search and the cited pages matter.
  • Paste the exact same wording every time. Changing one word changes the test.
  • Note the date, the assistant and the model name if the app shows it.

Step 3: Why ask each question three times?

The same question can return a different list a minute later. Assistants generate text with some randomness, and with web search on they may read different pages on each run. One answer is an anecdote. Three runs per question per assistant give you a pattern you can act on.

Ten questions, three assistants, three runs: 90 answers. Copying the brand names out of each takes about 20 seconds, so the whole test fits in about half an hour.

Step 4: What should you record?

Use one row per answer. A spreadsheet with these columns is enough:

ColumnWhat to write
QuestionThe exact wording you pasted
Assistant and runFor example "ChatGPT, run 2"
Brands namedAll brands, in the order they appear
Your positionYour place in that list, or "not named"
Cited pagesThe URLs the answer linked to as sources
What it said about youCopy the sentence, and mark it right or wrong

Step 5: How do you read the results?

Four numbers tell you most of what you need:

  1. Mention rate: the share of answers that name you at all. Named in 9 of 90 answers is 10%.
  2. Average position: when you are named, are you first or fifth? The first two names get most of the attention.
  3. Who replaces you: count how often each competitor appears. The brand named most often is the one the assistants treat as the default for your category.
  4. Cited sources: list every domain the answers linked to, and count them. These are the pages the assistants trust for your category.

The cited sources are the most useful part. They are usually comparison articles, review sites, directories and community threads. If a "best tools for X" article is cited in a third of the answers and does not mention you, getting onto that page is worth more than another post on your own blog.

Also read what the assistants say about you when they do name you. Wrong prices, old features or a confused description are a sign that the facts about your brand are inconsistent across the web.

What should you fix first if you are rarely named?

Work from the cheapest fix to the slowest:

  1. Check that AI crawlers can reach your site. A robots.txt rule or a firewall setting can block them without anyone noticing. Our free AI crawler checker tests this in a few seconds.
  2. Say plainly on your homepage what you are, who it is for and where you operate, in one sentence an assistant can quote.
  3. Publish honest comparison and "best for" pages for your category, since assistants lean on that format when buyers ask for options.
  4. Get listed on the sources from Step 5: the review sites, directories and articles the assistants already cite.

If the terms are new to you, our guide What is Generative Engine Optimization? explains the technical and content side in more detail.

What are the limits of a manual test?

Ninety answers is a small sample, so treat small differences as noise. Results also move when the assistants update their models or when new pages about your category appear. Run the same test once a month with the same questions, and compare the numbers, not single answers.

If you would rather not copy 90 answers into a spreadsheet, the AnswerSeat free audit runs buyer questions across ChatGPT, Gemini and Claude for you, counts who is named and where, checks the technical side and gives you a prioritised list of fixes.

Questions people also ask

How often should I check whether ChatGPT recommends my brand?

Once a month is enough for most businesses, plus once after any big change such as a new homepage, a rebrand or a new listing on a major review site. Keep the same questions so the results are comparable.

Does it matter if I am logged in when I test?

Yes. Saved memories and past chats can change the answer. Use a temporary chat or a fresh account with memory off to see something close to what a new buyer sees.

Why does ChatGPT give a different answer every time?

Assistants generate each answer with some randomness, and with web search on they may read different pages on each run. That is why you should ask each question at least three times and look at the pattern.

Do I need to test Gemini and Claude too, or only ChatGPT?

Test all three if you can. They read different sources and often name different brands, and your buyers do not all use the same assistant.

See which brands ChatGPT, Gemini and Claude name instead of you.

The free audit runs buyer questions for you and hands you a prioritised list of fixes.

Run my free audit

No credit card. Results in 3 to 5 minutes.

Sources

Comments

No comments yet. Questions and experiences are welcome.

Sign in to comment

Comments are open to AnswerSeat members. We email you a one-click sign-in link; new here, the same link creates your free account.

By continuing you agree to the Terms of Use and Privacy Policy.

Keep reading