Snoika
AI search visibility·Supporting article

AI mentions: what brand mention tracking shows, what it misses

AI mentions are the times an assistant such as ChatGPT or Google AI Overviews names your brand in an answer. The same prompt returns different lists of brands across runs, so tracking captures answers that differ from those your buyer sees. Model updates can also reset your baseline without notice. A mention count estimates direction and how you compare with competitors, with a margin of error.

Portrait of Artem LozinskyArtem LozinskyOperations and product, Snoika · 6 min read
Five people in their forties in a home office with a big window and wooden shelves.

Why mention counts mislead

A mention count works like a sample. In classic search, position 4 stayed position 4 until Google changed its results. An AI assistant writes a new answer every time someone asks, so a count only describes the runs you happened to record. After testing this, Rand Fishkin, CEO of SparkToro, wrote that "any tool that gives a 'ranking position in AI' is full of baloney." The rest of this page covers where the error comes from and what to do about each source. Check whether each figure in your current reports comes with the number of runs behind it.

Answer variance distorts counts

To handle answer variance, run each prompt many times and report the share of runs that name your brand. Language models build answers by sampling words from probabilities. Settings like temperature add randomness on purpose, so every answer is one draw from a range of possible answers. Fishkin and a team of 600 volunteers measured how wide that range gets. They ran 12 recommendation prompts nearly 3,000 times across three systems. Alongside ChatGPT and Claude, the tests covered Google's AI. The chance of getting the same brand list twice was under 1 in 100. Getting the same list in the same order was closer to 1 in 1,000.

Stat tiles: across nearly 3,000 runs, the chance of the same brand list twice was under 1 in 100
The chance of getting the same brand list twice was under 1 in 100. Source: SparkToro, Rand Fishkin.

Fishkin advised, "If you don't like an answer, or your brand doesn't show up where you want it to, just ask a few more times." So a mention rate across repeated runs is a fair measure.

Personalisation skews results

Personalization makes AI mention counts unreliable because the answer a tracker sees differs from the answer each buyer gets. On April 10, 2025, OpenAI announced that ChatGPT memory "can now reference all of your past chats to provide more personalized responses." A buyer who has spent months asking about one vendor brings that history into every new question. Location and language change the answer too, and so does whether the person is logged in.

Trackers query from clean sessions with no history. That gives you a stable view of the default answer. It's a fair benchmark, but it describes nobody's real experience. So treat tracked mentions as the new-user baseline, and ask your tool which location and language settings it uses.

Sampling gaps hide reach

A prompt set measures only the questions inside it, so any visibility outside the set never shows up in your data. Real buyers type long, specific questions in their own words, and no fixed list covers them all. Fishkin questioned how many prompts accurately represent a market.

The size of the set changes what you can conclude. Our prompt sets are generated from your content map, so the set grows with the pages you plan, and custom prompts can be added. Ten prompts can tell you whether an assistant knows your brand at all without measuring how much of your market it reaches. To check your coverage, sort your prompts by buyer stage and compare them with the queries in your Google Search Console data. Any stage with only one or two prompts is a blind spot.

Model versions shift baselines

A model update can move your mention rate even when nothing about your brand has changed. Researchers at Stanford and UC Berkeley compared the March and June 2023 versions of GPT-4. They found its accuracy on one task fell from 84% to 51%, and the product name stayed the same the whole time. Which brands a model recommends can drift in the same silent way.

Bar chart: GPT-4 accuracy on one task fell from 84% to 51% between March and June 2023
A silent model update can move results with no change to your brand. Source: Stanford and UC Berkeley researchers, 2023.

A drop caused by a model update says nothing about your content. Mark known release dates on your trend chart. When the line jumps on one of those dates, start a new baseline and avoid comparing across the break.

Reading mention data responsibly

Read GEO metrics such as mention data as a range that moves over time. A single number is the wrong frame. Two habits keep you honest.

How many runs make a figure stable

Stability comes from repetition. In the SparkToro study, each volunteer ran a prompt 60 to 100 times. That's a research-grade level. A weekly tracker running each prompt once or twice gives a rougher figure, and you get the stability back by averaging across weeks. A four-week rolling average smooths out most run-to-run noise.

How to report a change

Report changes larger than your normal weekly swing. If your mention rate wobbles between 10% and 15% from week to week, a move from 12% to 14% is noise. Snoika's founder Anton Vedeshin reported to Startup Reporter 1StopVAT's move from 2.7% to 28.3% ChatGPT visibility in 16 weeks. A shift that size over that long a period survives any reasonable margin of error. When you report to leadership, give the range and the run count with each number.

What share of voice signals

Share of voice is your slice of all brand mentions across a prompt set, compared with the competitors you chose to track. That makes it a relative measure. If you add a competitor to the list, your share falls even though your mentions haven't changed. It also says nothing about how the assistant describes you. A brand mentioned with a warning attached counts the same as one that gets recommended.

Use share of voice to measure whether you're gaining or losing ground against a fixed group of rivals. Keep the competitor list frozen for each reporting period. Show the absolute mention rate next to it, too. Share of voice can rise while your rate stays flat if a rival is losing ground.

Metrics that complement mentions

Other metrics, such as AI citation tracking, show the value of the mentions an assistant gives you. In onboarding, we ask for access to Google Analytics 4 (GA4) and Google Search Console so mention data sits next to traffic data. We also ask for access to Google Tag Manager.

Citation rate

What it shows
How often an answer links your pages as a source
Where it falls short
A page can be cited without your brand being named

Sentiment

What it shows
Whether the answer frames you well or adds caveats
Where it falls short
Judged on samples, so it varies as much as mentions do

AI referral traffic in GA4

What it shows
Visits that arrive from assistants
Where it falls short
Undercounts reach, because few users click through

Branded search volume in Search Console

What it shows
Whether AI exposure sends people to search for your name
Where it falls short
Lags behind mentions and mixes with other channels

Pew Research Center found that Google users clicked a link inside an AI summary in just 1% of visits. Traffic alone will make AI visibility look smaller than it is.

Operationalising AI visibility tracking

Tracking becomes reliable when you fix the method and change only the content you're testing. Our tracking uses weekly prompt sets to measure mentions and share of voice in ChatGPT and Google AI Overviews, where most AI search traffic happens. It also measures sentiment and runs a citation tracker.

  1. Write prompts from real buyer questions, with several prompts for each buyer stage.
  2. Lock and record the query settings for location and language. Keep sessions logged out and record that state too.
  3. Run the full set every week on the same day.
  4. Log mentions and citations as separate fields for every answer. Record sentiment tracking in its own field too.
  5. Mark model release dates on the timeline as they happen.
  6. Review the four-week averages once a month, with a person checking any answer behind a big swing.

The human review in Step 6 matters most because automated scoring can misread a list that names your brand in passing as a recommendation. At Snoika, a human review layer sits on top of the automated workflows for this reason.

Raising your AI mention rate

You get more AI mentions when more of the pages assistants read name your brand, both on your own site and elsewhere. A 2025 study of 75,000 brands found that branded web mentions correlated with AI Overview visibility at 0.664, against 0.218 for backlinks. About a quarter of the brands studied had no mentions at all.

Your own pages count when they directly answer the prompts you track. 3DPrinterOS recorded an 87% increase in LLM brand mentions (a client figure reported to Startup Reporter). A brand with pages in only one language gives an assistant nothing to cite when buyers ask in another, which is why we create content in 135+ languages and regional variants.

Next steps

Build your prompt set from the questions your buyers actually ask. Then run them unchanged for four weeks before you report a single figure, because that gives you the variance range everything else gets judged against. Keep a log of the model version and query settings for each run. It costs little now, and it's the only way to explain a sudden jump in your AI mentions later. Snoika audits websites and monitors brand visibility in Google and AI assistants, and a free report is the usual first step. Get a free AI visibility report to see your starting baseline.

FAQ

See how AI talks about your brand