An AI Citation Audit You Can Run Monthly

An AI citation audit is a repeatable check of whether answer engines reference your pages, run monthly with three free inputs: Google Search Console’s Generative AI performance report, a manual check of your Search Console AI inclusion setting, and a handful of your own target queries typed directly into ChatGPT, Perplexity, and Gemini with the results logged. None of the platforms publish a single “citation score,” and independent testing has found these tools get sourcing wrong at rates that make a one-time check unreliable, so the audit has to repeat on a schedule, not run once and get filed away.

I went looking for an “AI citation audit” tool and found a lot of blog posts with that phrase in the title and not much underneath it — a checklist repeated from post to post, no links back to anything a platform actually publishes. So I built the checklist myself, from documentation I could fetch and read, plus the two independent studies that have actually tested citation accuracy with a transparent method. This is what’s verifiable today, including the parts that aren’t flattering.

What the platform documentation actually says

Two Google Search Console pages matter here, and a third data point comes from reading each AI platform’s own help documentation rather than secondary summaries of it.

First, Search Console has a real report for this, and it’s narrower than people assume. The Generative AI performance report shows impressions, clicks, and average position for queries that triggered AI Overviews or AI Mode where your site appeared. Google’s own documentation flags a known data gap — an anomaly affecting August 13–17, 2025 — so treat the numbers as directionally useful rather than exhaustive. This report, not a separate AI-specific dashboard, is the starting point for any audit on a Google-indexed site.

Second, there’s a setting worth checking before you conclude a page has a citation problem. The Search Generative AI control page documents two states: Include, the default, where content “may appear and be used to help ground AI-powered responses,” and Exclude, where it explicitly will not. The same page states that excluding a page “doesn’t act as a signal for ranking purposes elsewhere on Search” and doesn’t change participation in services like Merchant Center or Ads — it is a single, narrow toggle, not a visibility penalty that bleeds into other features. If a page you expect to see in AI Overviews never shows up, this setting is a five-minute thing to rule out before assuming the content itself is the issue.

Third, OpenAI and Perplexity each describe their own mechanics in ways that double as audit instructions. OpenAI’s help documentation states that ChatGPT’s search results “may be incomplete, outdated, or otherwise inaccurate” and that no particular placement is promised — and separately confirms that OAI-SearchBot needs crawl access before a page is even eligible to be surfaced, which is a prerequisite worth checking before assuming the problem is citation behavior rather than crawl access. Perplexity’s help center describes its answers as built from “in-line, numbered citations” tied to retrieved sources, which means the audit step for Perplexity is literally running your target query and checking whether your domain appears among the numbered citations in the response. Neither platform publishes a lookup tool for this. You have to run the query and read the answer.

What the independent evidence supports

The reason a one-time check isn’t enough is that the tools themselves are inconsistent, and two independent studies from Columbia Journalism Review’s Tow Center for Digital Journalism measured that inconsistency directly rather than asserting it.

In November 2024, researchers tested 200 direct quotes drawn from 20 publishers against ChatGPT Search and found it confidently wrong in 146 of those attempts — 73% of the sample. Results were inconsistent across repeated, identical prompts, meaning the same query could return a different, equally confident answer on a different run. OpenAI pushed back on the testing method as atypical of normal usage and said citation accuracy has since improved; I’m noting that pushback rather than treating the study as the final word on current performance.

A March 2025 follow-up from the same researchers tested eight AI search tools — ChatGPT, Perplexity, Perplexity Pro, Grok 3, Grok 3 (Think), Copilot, Gemini, and DeepSearch — and found that collectively, more than 60% of answers were incorrect. The spread between tools was wide: Perplexity’s error rate was 37%, Grok 3’s was 94%. ChatGPT misidentified the source article in 134 of 200 tests and signaled low confidence only 15 times, never once declining to answer outright. Copilot was the single tool in the study that declined to answer more often than it answered incorrectly. And in a detail worth sitting with if you’re choosing a premium subscription expecting more reliable citations, the paid tiers — Perplexity Pro and Grok 3 — had higher error rates than their free counterparts, because they answered definitively more often instead of declining when uncertain.

None of that is an argument against trying to earn AI citations. It’s an argument against trusting a single check, on a single tool, on a single day, as a verdict on whether your content “works” for AI answer engines.

A practical monthly checklist

This is the audit I’m actually running, repeated on a recurring monthly date so the comparison across months means something.

  • Pull the Generative AI performance report in Search Console and record impressions, clicks, and average position for your target pages — even if every number is zero. Zero is data, not an error state.
  • Confirm the Search Generative AI control setting is on Include for pages you want in AI Overviews, so you’re not auditing a page you’ve accidentally excluded.
  • Pick 5–10 real target queries and run each one manually in ChatGPT, Perplexity, and Gemini. Record whether your domain is cited, not cited, or cited incorrectly — a wrong citation is a different problem than no citation.
  • If a query returns a wrong or outdated citation, check whether the underlying page needs a direct, extractable opening paragraph; the pattern I use for that is covered in writing answer capsules.
  • Don’t add schema markup specifically chasing this. I checked the documentation on that question directly in structured data in the age of AI answers and separately in does schema markup help AI citations, and neither Google nor OpenAI documents a markup requirement for AI visibility.
  • If you want a longer-running measurement method rather than a monthly spot-check, see how to measure AI citations, which this audit is meant to complement, not replace.
  • Keep a dated log — a spreadsheet row per month is enough. The value of this audit is the trend across five or six months, not any single month’s result.

If you’re trying to decide how much time this deserves relative to classic SEO work, the distinctions in GEO vs SEO vs AEO are worth reading first — AI citation visibility is a real, separate thing to track, but it sits alongside organic search work, not above it.

The honest limits of current evidence

I’ll report this site’s own numbers instead of skipping past them. In Google Analytics, across sessions from July 1 through October 5, 2026, no AI-assistant referrer domain — not chatgpt.com, not perplexity.ai, not claude.ai, not copilot.microsoft.com, not gemini.google.com — appears anywhere in the session-source breakdown. The top sources in that window were direct traffic at 171 sessions, Google search at 6, and GitHub referrals at 4. In Search Console, zero impressions showed up for any query containing the word “citation” over the same window. There is no existing AI-citation visibility on this site to report as a success story. This post is the starting point for the audit, not a recap of one that already worked.

There are gaps in what any of the platforms will tell you, too. No platform publishes a general, cross-site citation-rate figure you could look up for your own domain — the numbers in the Tow Center studies are results from a specific test set of 200 quotes and a specific list of tools, not a live, ongoing score. Treat the two studies as dated findings about how these systems behaved when tested, not as a current, continuously updated grade. And excluding a page from AI features, per Google’s own documentation, doesn’t carry over as a ranking signal elsewhere — so if you’re troubleshooting a page that’s missing from AI Overviews, that setting is a thing to check and rule out, not a universal explanation for low AI visibility.

So: run the checklist above once a month, log what you find including the zeros, and treat any single check — a good one or a bad one — as one data point in a series rather than a verdict.


Shivaa Tripathi runs organic-os, an experiment in publishing SEO and AEO work with the research and verification shown rather than asserted.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *