Skip to content
User fetcher Run by Anthropic

What is Claude-User?

Short answer: Claude-User is the agent Claude uses when a person asks it something that needs a web page. Anthropic says blocking it prevents Claude retrieving your content in response to a user query, which may reduce your visibility for user directed web search. Unlike some other companies' user agents, Anthropic says its bots honour robots.txt.

Claude-User at a glance

Run by
Anthropic
Type
User fetcher
Used for
Pages Claude users ask about
robots.txt name
Claude-User
Follows robots.txt
Yes
Last checked
18 November 2026
User agent
Claude-User Anthropic names the bot but does not list a full user agent string on its help page.

What does Claude-User do?

Anthropic says Claude-User supports Claude users: when people ask Claude questions, it may access websites using the Claude-User agent.

Anthropic's help page states that its bots, which include Claude-User, respect robots.txt. That differs from OpenAI's ChatGPT-User and Perplexity-User, whose operators say robots.txt may not apply to user initiated fetches.

What happens if you block Claude-User?

Anthropic says disabling Claude-User prevents its system from retrieving your content in response to a user query, which may reduce your site's visibility for user directed web search.

Blocking Claude-User does not change training (ClaudeBot) or Claude's search index (Claude-SearchBot).

How do you allow or block Claude-User in robots.txt?

Does Claude-User follow robots.txt? Yes. Anthropic says its bots honour industry standard robots.txt directives, support the non standard Crawl-delay rule, and do not try to bypass CAPTCHAs. Add the rule to the robots.txt of every subdomain you want to cover.

Add one of these groups to the robots.txt file at the root of your site. Several User-agent lines in a row share the rules below them, and a crawler follows only the most specific group that names it, so repeat any private paths in that group.

Block Claude-User

User-agent: Claude-User
Disallow: /

Allow Claude-User, keep /admin/ private

User-agent: Claude-User
Allow: /
Disallow: /admin/

Slow Claude-User down instead of blocking it

User-agent: Claude-User
Crawl-delay: 1

Should you block Claude-User?

Our view: Leave it allowed. These fetches happen when someone asks Claude about your page or topic, which is exactly the moment you want to be readable.

How do you verify a real Claude-User request?

Anthropic publishes the IP addresses its bots use at claude.com/crawling/bots.json (one list for all its bots). It advises against blocking those IP addresses to opt out, because the bots may then be unable to read your robots.txt.

Claude-User questions

What is the Claude-User user agent?
Requests contain Claude-User in the user agent. Anthropic names the bot but does not list a full user agent string on its help page.
Does Claude-User follow robots.txt?
Yes. Anthropic says its bots honour industry standard robots.txt directives, support the non standard Crawl-delay rule, and do not try to bypass CAPTCHAs. Add the rule to the robots.txt of every subdomain you want to cover.
Does Claude-User respect robots.txt?
Anthropic says its bots honour industry standard robots.txt directives, and it lists Claude-User among them.
What is the difference between Claude-User and Claude-SearchBot?
Claude-User fetches a page when a user's request needs it; Claude-SearchBot indexes content in advance to improve Claude's search results.
Will blocking Claude-User stop training on my content?
No. Training is controlled with ClaudeBot.
How do I check whether my site allows Claude-User?
Run your domain through the free ONEGO AI crawler checker. It reads your robots.txt and shows whether Claude-User and 15 other AI crawlers are allowed, blocked or partly restricted, and what each rule means.

Crawlers can read you. Does AI recommend you?

See your ONE Score first · 7 day free trial, no card needed · Pay only if you keep tracking