What is Claude-User?
Short answer: Claude-User is the agent Claude uses when a person asks it something that needs a web page. Anthropic says blocking it prevents Claude retrieving your content in response to a user query, which may reduce your visibility for user directed web search. Unlike some other companies' user agents, Anthropic says its bots honour robots.txt.
Claude-User at a glance
- Run by
- Anthropic
- Type
- User fetcher
- Used for
- Pages Claude users ask about
- robots.txt name
-
Claude-User - Follows robots.txt
- Yes
- Last checked
- 18 November 2026
- User agent
-
Claude-UserAnthropic names the bot but does not list a full user agent string on its help page.
What does Claude-User do?
Anthropic says Claude-User supports Claude users: when people ask Claude questions, it may access websites using the Claude-User agent.
Anthropic's help page states that its bots, which include Claude-User, respect robots.txt. That differs from OpenAI's ChatGPT-User and Perplexity-User, whose operators say robots.txt may not apply to user initiated fetches.
What happens if you block Claude-User?
Anthropic says disabling Claude-User prevents its system from retrieving your content in response to a user query, which may reduce your site's visibility for user directed web search.
Blocking Claude-User does not change training (ClaudeBot) or Claude's search index (Claude-SearchBot).
How do you allow or block Claude-User in robots.txt?
Does Claude-User follow robots.txt? Yes. Anthropic says its bots honour industry standard robots.txt directives, support the non standard Crawl-delay rule, and do not try to bypass CAPTCHAs. Add the rule to the robots.txt of every subdomain you want to cover.
Add one of these groups to the robots.txt file at the root of your site. Several User-agent lines in a row share the rules below them, and a crawler follows only the most specific group that names it, so repeat any private paths in that group.
Block Claude-User
User-agent: Claude-User
Disallow: / Allow Claude-User, keep /admin/ private
User-agent: Claude-User
Allow: /
Disallow: /admin/ Slow Claude-User down instead of blocking it
User-agent: Claude-User
Crawl-delay: 1 Should you block Claude-User?
Our view: Leave it allowed. These fetches happen when someone asks Claude about your page or topic, which is exactly the moment you want to be readable.
How do you verify a real Claude-User request?
Anthropic publishes the IP addresses its bots use at claude.com/crawling/bots.json (one list for all its bots). It advises against blocking those IP addresses to opt out, because the bots may then be unable to read your robots.txt.
Related AI crawlers
-
Claude-SearchBotAnthropic · Search crawler Indexes pages to improve the search results Claude gives users. -
ClaudeBotAnthropic · Training crawler Collects content that may be used to train Anthropic's AI models. -
ChatGPT-UserOpenAI · User fetcher Opens a page when a ChatGPT user asks it to, for example to read a link. -
Perplexity-UserPerplexity · User fetcher Fetches a page when a Perplexity user's question needs it.
See every AI crawler in one table, check your own robots.txt with the free AI crawler checker, or give AI tools a clean summary of your site with the llms.txt generator.
Claude-User questions
What is the Claude-User user agent?
Does Claude-User follow robots.txt?
Does Claude-User respect robots.txt?
What is the difference between Claude-User and Claude-SearchBot?
Will blocking Claude-User stop training on my content?
How do I check whether my site allows Claude-User?
Crawlers can read you. Does AI recommend you?
See your ONE Score first · 7 day free trial, no card needed · Pay only if you keep tracking