My own access logs show three different AI bot behaviors at work, and optimizing for one does not necessarily move another.
Training crawlers. ClaudeBot, GPTBot, Amazonbot. They pull broadly to feed model training and large index builds. High volume. Tells you the AI vendors know your site exists.
Search index crawlers. Claude-SearchBot, OAI-SearchBot, and similar. They populate the search layer AI assistants reach into when answering live questions. Lower volume, more selective.
User fetch. When a real user asks an assistant a question and the assistant pulls a specific page in real time, the request comes through a different user agent. Claude-User and ChatGPT-User are the two I see most often hitting my insights pages on AI and automation. This is the behavior closest to a citation, because there is an actual user on the other side of the fetch.
Across roughly a few months of server logs, I have not seen a clean correlation between the volume of training crawler activity and the volume of user fetches. Crawlers grab new content aggressively. Assistants answering real user questions are much more selective.
The implication for anyone investing in GEO is uncomfortable. "A training crawler hit my site 300 times last month" is not the same indicator as "a real user fetched my page through an assistant." Optimizing for crawl frequency is not necessarily optimizing for what real users see.
If your marketing depends on AI assistants surfacing your content to real users, the user fetch is the number to watch. Training crawler volume confirms the AI vendors know you exist. A user fetch puts a real person on the other end of a question.
Does a user fetch translate into a referral click, a lead, or a customer? If you have found a clean way to tie user fetches to downstream business, I would like to hear how.
Written by Duane Grey
AI Strategy & Implementation
Independent AI consultant helping companies cut through hype and deploy systems that produce real results.