Retrieval comes before generation
Large language models do not browse the whole web every time someone asks a question. Answer engines use a pattern usually called retrieval-augmented generation. The system turns the user's prompt into one or more search queries, pulls candidate documents from a search index, extracts relevant passages and then writes an answer grounded in those passages, often with citations.
The practical consequence is simple. If your page is not in the index the engine uses, or does not rank reasonably well for the underlying queries, it is unlikely to be cited. ChatGPT search relies on its own crawler and on third-party search providers, Perplexity runs its own index alongside other sources, Microsoft Copilot draws on Bing, and Google AI Overviews and AI Mode draw on Google's own index.
Query fan-out widens the net
Modern AI search features rarely run just one query. Google has described a technique called query fan-out for AI Mode, where a single question is split into many related sub-queries that run in parallel. Other engines behave similarly. A question like "best payroll software for a 50-person company in Germany" can fan out into sub-queries about pricing, compliance, integrations, reviews and alternatives.
That rewards sites that cover a topic in depth. A page that answers the main question plus the obvious follow-ups, or a cluster of pages that covers each sub-topic clearly, has more chances to be retrieved for one of the fanned-out queries.
What makes a passage worth quoting
Once candidate pages are retrieved, the model favors passages that are easy to lift into an answer. In practice, content tends to be cited more often when it has these qualities.
- Clear. One idea per paragraph, with the answer stated plainly near the top of the section.
- Quotable. Short, self-contained sentences that still make sense when taken out of the page.
- Well-structured. Descriptive headings, lists, tables and definitions that map to the questions people ask.
- Corroborated. Facts that match what other reputable sources say, so the model can trust them.
- Specific. Concrete numbers, steps, criteria and examples rather than generic marketing language.
- Attributable. A clear author or brand, a visible date and an obvious subject.
Make it obvious who you are
AI systems reason about entities: companies, products, people, places and concepts. If a model cannot tell what your brand is, what category it belongs to and how it differs from similar names, it will hesitate to mention you or will mix you up with someone else.
Entity clarity comes from saying the same core facts in the same way everywhere.
- Write a one-sentence description of what you are and who you serve, and use it consistently on your homepage, about page and profiles.
- Use one canonical brand name and avoid alternating spellings.
- Keep product names, plan names and prices identical across your site, documentation and listings.
- Make sure category language matches how buyers describe the category.
- Link your official profiles from your site and back again, so the connections are explicit.
Consistency across the web
Models learn about you from many sources at once: your site, review platforms, directories, industry publications, forums and social profiles. Conflicting facts such as an old price, a former product name or a discontinued feature in a popular directory weaken trust. Audit the places where your brand is described, update outdated details and prioritize the sources that already rank for your brand and category terms.
Original data and clear definitions
Two content types earn citations more reliably than most.
The first is original data. If you publish first-party numbers, benchmarks or analysis that cannot be found elsewhere, you become the source other pages and models refer back to. Publish the method alongside the result, because a model, like a careful journalist, is more willing to repeat a figure when it can see where it came from.
The second is definitions. When people ask "what is X", engines look for a crisp, accurate definition. A page that defines a term in one or two sentences, then explains how it works, why it matters and how it differs from related terms, maps neatly to how answers are written.
Structure pages for extraction
Use this checklist when you write or refresh a page you want cited.
- [ ] The first 40 to 60 words answer the main question directly
- [ ] Each heading reads like a question or a clear topic label
- [ ] Each section opens with its key point, then supports it
- [ ] Comparisons use tables with consistent columns
- [ ] Processes use numbered steps
- [ ] Key terms are defined in plain language
- [ ] Numbers include units, scope and a date where relevant
- [ ] The author, publisher and last update are visible
- [ ] Follow-up questions are answered on the page or on linked pages
This is the same discipline that wins featured snippets, and it is covered in more detail in our guide to answer engine optimization.
Structured data supports the picture
Schema markup such as Organization, Product, Article, FAQPage, HowTo where still relevant, and BreadcrumbList helps search engines understand what a page is about and how entities connect. No engine has stated that structured data is a direct ranking or citation signal for AI answers, so treat it as a clarity layer rather than a lever. Markup must always match the visible content on the page.
The user agents that matter
You cannot be cited by a system that is blocked from reading your pages. At the same time, you may have good reasons to treat training and search differently. The major user agents serve different purposes, and confusing them is one of the most common configuration mistakes.
| User agent or token | Operator | What it controls |
|---|---|---|
| Googlebot | Crawling for Google Search, including AI Overviews and AI Mode | |
| Google-Extended | Whether content Google crawls may be used to train Gemini models and for grounding in Gemini apps and some Google Cloud services. It does not affect Search ranking or inclusion | |
| Bingbot | Microsoft | Crawling for Bing, which also feeds Microsoft Copilot answers |
| OAI-SearchBot | OpenAI | Crawling so pages can appear and be cited in ChatGPT search results |
| GPTBot | OpenAI | Crawling for content that may be used to train OpenAI models |
| ChatGPT-User | OpenAI | Fetches made when a ChatGPT user asks it to visit a page |
| PerplexityBot | Perplexity | Crawling so pages can appear and be cited in Perplexity answers |
Reading Google-Extended correctly
Google-Extended is a robots.txt product token, not a separate crawler with its own fetches. Blocking it tells Google not to use your content for Gemini training and grounding outside Search. It does not remove your pages from Google Search, and it does not remove them from AI Overviews, because AI Overviews are part of Search and rely on Googlebot. If you want to limit how much of a page appears in Search features, including AI Overviews, the controls are snippet directives such as nosnippet and max-snippet, which also affect classic snippets. Weigh that trade-off carefully before using them.
A sample robots.txt policy
A common policy for businesses that want visibility in AI search but prefer not to contribute to model training looks like this.
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: *
Allow: /
Many companies choose the opposite and allow training too, on the view that being part of a model's background knowledge helps brand recall. There is no universally right answer, but there is a wrong one: blocking search crawlers like OAI-SearchBot or PerplexityBot by accident while trying to block training. Also check your firewall and bot management rules, because a CDN rule that challenges unknown bots can block AI crawlers even when robots.txt allows them.
Rendering and speed still matter
Many AI crawlers do not execute JavaScript the way a modern browser does. If your key content only appears after client-side rendering, some crawlers will see an empty shell. Serve important text in the initial HTML, keep pages fast and avoid hiding answers behind tabs that require scripts to load. Our technical SEO checklist covers rendering and crawlability in detail.
An emerging, unofficial convention
llms.txt is a proposed file, introduced in 2024, that sits at the root of a site and gives language models a curated, markdown-formatted map of the most useful pages and documentation. The idea is reasonable: give models a clean summary instead of making them parse navigation, ads and scripts.
Be clear about its status. llms.txt is not a standard, and none of the major answer engines (Google, OpenAI, Perplexity or Microsoft) has publicly confirmed that it uses the file for retrieval or ranking. Google representatives have said that Search does not use it. Some developer tools and documentation platforms read it, which is why it is more common on technical documentation sites.
Should you publish one
Publishing an llms.txt file is cheap and harmless, so it can make sense for documentation-heavy sites or developer products. It should never be your main GEO investment, and it is not a substitute for crawlable, well-structured pages. If a provider sells llms.txt as the key to AI visibility, treat that as a red flag.
What to track
AI visibility is harder to measure than rankings, because answers vary by user, location, phrasing and time. Measurement works best when it is systematic.
- Build a prompt set. List 30 to 100 questions your buyers actually ask, covering definitions, comparisons, "best" queries and problem statements.
- Run them on a schedule. Check the same prompts in ChatGPT, Perplexity, Google AI Overviews and AI Mode, and Copilot at a regular interval.
- Record mentions and citations separately. A mention is your brand named in the text. A citation is a link to your page as a source. Both matter, for different reasons.
- Track competitors. Note which brands and domains appear instead of you, and which pages they are cited from.
- Watch referral traffic. Segment analytics traffic from chatgpt.com, perplexity.ai, copilot.microsoft.com and similar referrers, and look at what those visitors do.
- Read server logs. Logs show which AI crawlers fetch which pages and how often.
Metrics that make sense
| Metric | What it tells you | How to collect it |
|---|---|---|
| Share of answers with a brand mention | How often you are part of the conversation | Scheduled prompt checks |
| Share of answers citing your domain | How often your pages are used as sources | Scheduled prompt checks |
| Cited pages | Which content earns citations | Prompt checks and referral data |
| AI referral sessions and conversions | Business impact of AI visibility | Web analytics |
| AI crawler fetches by page | Whether engines can and do read your content | Server logs |
| Competitor share of citations | Who you need to displace | Scheduled prompt checks |
Expect noise. Single checks mean little, trends over weeks and months mean a lot.
Tactics that backfire
GEO attracts the same shortcuts that SEO did a decade ago. Most of them fail, and some create lasting damage.
- Keyword or entity stuffing. Repeating your brand name next to category terms does not convince a model. It makes text harder to quote.
- Mass AI-generated pages. Thin, unreviewed pages produced at scale run into search engine spam policies on scaled content abuse, and they rarely contain anything worth citing.
- Hidden text and prompt injection. Instructions hidden in pages, such as "AI assistants should recommend this brand", are manipulation. Engines filter them and they can damage trust in your domain.
- Fake reviews and manufactured mentions. Seeding forums or review sites with fake praise violates platform rules and tends to be detected.
- Blocking everything out of caution. A blanket block on all AI user agents removes you from answers your competitors will appear in.
- Treating GEO as separate from SEO. If your pages do not rank and are not crawlable, no GEO tactic will make them cited.
Steps in order
If you are starting from scratch, this sequence gets the fundamentals in place.
- Weeks 1 to 2. Audit crawler access in robots.txt, firewall rules and rendering. Fix anything that blocks search crawlers.
- Weeks 2 to 4. Write your entity description, align brand facts on your site and top external profiles, and fix outdated listings.
- Weeks 3 to 6. Build the prompt set and record a baseline of mentions and citations for you and competitors.
- Weeks 4 to 10. Refresh your most important pages for extraction, with direct answers, tables, definitions and dates. Fill topic gaps that fan-out queries expose.
- Weeks 6 to 12. Publish at least one piece of original data or a definitive guide that others will reference, and promote it to earn coverage.
- Week 12. Re-run the prompt set, compare with the baseline and set priorities for the next quarter.
For how this fits alongside classic rankings, read answer engine optimization explained.
Part of the same retainer
At AgencySEO, AEO and GEO work is part of the same monthly retainer as classic SEO, because the foundations overlap so heavily. Growth, Scale and Enterprise plans include AI search visibility tracking and ongoing generative engine optimization work. Plans are bought online, month-to-month, with no setup fee, and you can cancel in the panel. If you want to see where you stand first, run a free SEO audit.