Main Menu
🍁 Fall Sale: Save 25% on Pro Plugins & Books w/ code FALL2026
Perishable Press
Web Dev + WordPress + Security
Banhammer
Banhammer: Protect your WordPress site against threats.
Related Posts
+ More related posts »

Ultimate Block List to Stop AI Bots

♦ Posted by in .htaccess, Security, SEO
Updated • 58 comments

More than you might think, AI (Artificial Intelligence) and ML (Machine Learning) bots are crawling your site and scraping your content. They are collecting and using your data to train software like ChatGPT, OpenAI, DeepSeek, and thousands of other AI creations. Whether you or anyone approves of all this is not my concern for this post. The focus of this post is aimed at website owners who want to stop AI bots from crawling their web pages, as much as possible. To help people with this, I’ve been collecting data and researching AI bots for many months now, and have put together a “Mega Block List” to help stop AI bots from devouring your content.

The ultimate block list for stopping AI bots from crawling your site.

Contents

If you can edit a file, you can block a ton of AI bots.

Thanks: Special Thanks to Kristina Ponting for help with researching AI bots and sharing with the community. Find Kristina at Teskedsgumman and on Github.

Block AI Bots via BBQ Pro

🔥 Users of my BBQ Pro firewall plugin can add the Ultimate AI Block List with just a few clicks »

Block AI Bots via robots.txt

The easiest way for most website owners to block AI bots, is to append the following list to their site’s robots.txt file. There are many resources explaining the robots.txt file, and I encourage anyone not familiar to take a few moments to learn more.

In a nutshell, the robots.txt file is a file that contains rules for bots to obey. So you can add rules that limit where bots can crawl, whether individual pages or the entire site. Once you have added some rules, simply upload the robots file to the public root directory of your website. For example, here is my robots.txt for Perishable Press.

Using WordPress? Block bad bots automatically with my free plugin, Blackhole for Bad Bots. Trap bad bots in a virtual black hole :)

To block AI bots via your site’s robots.txt file, append the following rules. Understand that bots are not required to obey robots.txt rules. Robots rules are merely suggestions. Good bots will follow the rules, bad bots will ignore the rules and do whatever they want. To force compliance, you can add blocking rules via Apache/.htaccess. That in mind, here are the robots rules to block AI bots..

Blocks over 700+ AI bots and user agents.

Block list for robots.txt

Before using, read the Notes and Disclaimer.

# Ultimate AI Block List v1.8 20260105
# https://perishablepress.com/ultimate-ai-block-list/

# Allow all other bots full access
User-agent: *
Disallow:

# Block AI bots from all access
User-agent: .ai
User-agent: -ai
User-agent: _ai
User-agent: ai.
User-agent: ai-
User-agent: ai_
User-agent: ai=
User-agent: AddSearchBot
User-agent: Agentic
User-agent: AgentQL
User-agent: Agent 3
User-agent: Agent API
User-agent: Agent-
User-agent: AI Agent
User-agent: AI Article Writer
User-agent: AI Assistant
User-agent: AI Chat
User-agent: AI Content Detector
User-agent: AI Detection
User-agent: AI Dungeon
User-agent: AI Journalist
User-agent: AI Legion
User-agent: AI RAG
User-agent: AI Search
User-agent: AI SEO Crawler
User-agent: AI Training
User-agent: AI Web
User-agent: AI Writer
User-agent: AI2
User-agent: AIBot
User-agent: aiHitBot
User-agent: AIMatrix
User-agent: Airesearch
User-agent: AISearch
User-agent: AITraining
User-agent: Alexa
User-agent: Alice Yandex
User-agent: AliGenie
User-agent: AliyunSec
User-agent: Alpha AI
User-agent: AlphaAI
User-agent: Amazon
User-agent: Amelia
User-agent: AndersPinkBot
User-agent: AndiBot
User-agent: Anomura
User-agent: Anonymous AI
User-agent: Anthropic
User-agent: AnyPicker
User-agent: Anyword
User-agent: Applebot
User-agent: Aria AI
User-agent: Aria Browse
User-agent: Articoolo
User-agent: Ask AI
User-agent: Atlassian
User-agent: AutoGen
User-agent: Automated Writer
User-agent: AutoML
User-agent: Autonomous RAG
User-agent: AutoRAG
User-agent: Awario
User-agent: AWS Trainium
User-agent: Azure
User-agent: BabyAGI
User-agent: BabyCatAGI
User-agent: BardBot
User-agent: Basic RAG
User-agent: Bedrock
User-agent: Big Sur
User-agent: Bigsur
User-agent: Botsonic
User-agent: Brightbot
User-agent: Browser MCP Agent
User-agent: Browser Use
User-agent: BuddyBot
User-agent: Bytebot
User-agent: ByteDance
User-agent: Bytespider
User-agent: CarynAI
User-agent: CatBoost
User-agent: CC-Crawler
User-agent: CCBot
User-agent: Chai
User-agent: Channel3Bot
User-agent: Character
User-agent: Charstar AI
User-agent: Chatbot
User-agent: Chatsonic
User-agent: ChatUser
User-agent: Chinchilla
User-agent: Claude
User-agent: ClearScope
User-agent: Clearview
User-agent: Cognitive AI
User-agent: Cohere
User-agent: Common Crawl
User-agent: CommonCrawl
User-agent: Compass
User-agent: Content Harmony
User-agent: Content King
User-agent: Content Optimizer
User-agent: Content Samurai
User-agent: ContentAtScale
User-agent: ContentBot
User-agent: Contentedge
User-agent: ContentShake
User-agent: Conversion AI
User-agent: Copilot
User-agent: CopyAI
User-agent: Copymatic
User-agent: Copyscape
User-agent: CoreWeave
User-agent: Corrective RAG
User-agent: Cotoyogi
User-agent: CRAB
User-agent: Crawl4AI
User-agent: CrawlQ AI
User-agent: Crawlspace
User-agent: Crew AI
User-agent: CrewAI
User-agent: Crushon AI
User-agent: Cypex
User-agent: DALL-E
User-agent: DarkBard
User-agent: DarkBERT
User-agent: DataFor
User-agent: DataProvider
User-agent: Datenbank Crawler
User-agent: DeepAI
User-agent: Deep AI
User-agent: DeepL
User-agent: DeepMind
User-agent: Deep Research
User-agent: DeepResearch
User-agent: DeepSeek
User-agent: Devin
User-agent: Diffbot
User-agent: Dolma
User-agent: Doubao AI
User-agent: DuckAssistBot
User-agent: DuckDuckGo Chat
User-agent: DuckDuckGo-Enhanced
User-agent: Echobot
User-agent: Echobox
User-agent: Elixir
User-agent: FacebookBot
User-agent: FacebookExternalHit
User-agent: Factset
User-agent: Falcon
User-agent: FIRE-1
User-agent: Firebase
User-agent: Firecrawl
User-agent: Flyriver
User-agent: Frase AI
User-agent: FriendlyCrawler
User-agent: Gato
User-agent: Gemini
User-agent: Gemma
User-agent: Gen AI
User-agent: GenAI
User-agent: Generative
User-agent: Genspark
User-agent: Gentoo-chat
User-agent: Ghostwriter
User-agent: GigaChat
User-agent: GLM
User-agent: GodMode
User-agent: Goose
User-agent: GPT
User-agent: Grammarly
User-agent: Grendizer
User-agent: Grok
User-agent: Groq
User-agent: GT Bot
User-agent: GTBot
User-agent: GTP
User-agent: Hemingway Editor
User-agent: Hetzner
User-agent: Hugging
User-agent: Hunyuan
User-agent: Hybrid Search RAG
User-agent: Hypotenuse AI
User-agent: iAsk
User-agent: IbouBot
User-agent: ICC-Crawler
User-agent: Ideogram
User-agent: ImageGen
User-agent: Imagen
User-agent: ImagesiftBot
User-agent: ImageSpider
User-agent: img2dataset
User-agent: imgproxy
User-agent: INK Editor
User-agent: INKforall
User-agent: Instructor
User-agent: IntelliSeek
User-agent: Inferkit
User-agent: ISSCyberRiskCrawler
User-agent: Janitor AI
User-agent: Jasper
User-agent: Jenni AI
User-agent: Julius AI
User-agent: Kafkai
User-agent: Kaggle
User-agent: Kangaroo
User-agent: Keyword Density AI
User-agent: Kimi
User-agent: Klaviyo
User-agent: Knowledge
User-agent: KomoBot
User-agent: Kruti
User-agent: KunatoCrawler
User-agent: LAIONDownloader
User-agent: LangChain
User-agent: Langfuse
User-agent: Language AI
User-agent: Le Chat
User-agent: Lensa
User-agent: Leonardo
User-agent: Lightpanda
User-agent: LightRAG
User-agent: LinerBot
User-agent: LinkupBot
User-agent: LLaMA
User-agent: LCC
User-agent: LLM
User-agent: Local RAG Agent
User-agent: Lovable
User-agent: Magistral
User-agent: magpie-crawler
User-agent: MAI-Image
User-agent: Manus
User-agent: MarketMuse
User-agent: MBZUAI
User-agent: Meltwater
User-agent: Meta-AI
User-agent: Meta-External
User-agent: Meta-Webindexer
User-agent: Meta AI
User-agent: MetaAI
User-agent: MetaTagBot
User-agent: Middleware
User-agent: Midjourney
User-agent: Mini AGI
User-agent: MiniMax
User-agent: Mintlify
User-agent: Mistral
User-agent: Mixtral
User-agent: model-training
User-agent: Molmo
User-agent: Monica
User-agent: Nano
User-agent: Narrative
User-agent: NeevaBot
User-agent: netEstate
User-agent: Neural Text
User-agent: NeuralSEO
User-agent: NinjaAI
User-agent: NodeZero
User-agent: NotebookLM
User-agent: Nova Act
User-agent: NovaAct
User-agent: Nytheon
User-agent: OAI-SearchBot
User-agent: OAI SearchBot
User-agent: OASIS
User-agent: Olivia
User-agent: OLM
User-agent: Omgili
User-agent: Onyx
User-agent: Open AI
User-agent: Open Interpreter
User-agent: OpenAGI
User-agent: OpenAI
User-agent: OpenBot
User-agent: OpenInterpreter
User-agent: OpenPi
User-agent: OpenRouter
User-agent: OpenText AI
User-agent: Operator
User-agent: Outwrite
User-agent: OVHcloud
User-agent: Owlin
User-agent: Page Analyzer AI
User-agent: PaLM 2
User-agent: PanguBot
User-agent: Panscient
User-agent: Paperlibot
User-agent: Paraphraser.io
User-agent: peer39_crawler
User-agent: Perflexity
User-agent: Perplexity
User-agent: Petal
User-agent: Phind
User-agent: PiplBot
User-agent: Pixmo
User-agent: Playwright
User-agent: PoeBot
User-agent: PoeSearchBot
User-agent: Poggio
User-agent: Poseidon
User-agent: ProWritingAid
User-agent: Proximic
User-agent: PulsarRPA
User-agent: Puppeteer
User-agent: Python AI
User-agent: Python lxml
User-agent: PyTorch
User-agent: Qualified
User-agent: Quark
User-agent: QuillBot
User-agent: Qopywriter
User-agent: Qwen
User-agent: RAG Agent
User-agent: RAG Azure AI
User-agent: RAG Chatbot
User-agent: RAG Database
User-agent: RAG IS
User-agent: RAG Pipeline
User-agent: RAG Search
User-agent: RAG with
User-agent: RAG-
User-agent: RAG_
User-agent: Raptor
User-agent: React Agent
User-agent: Reasoning
User-agent: Redis AI RAG
User-agent: Reka Flash
User-agent: Replicate
User-agent: RobotSpider
User-agent: RunPod
User-agent: Rytr
User-agent: SaplingAI
User-agent: SBIntuitionsBot
User-agent: Scala
User-agent: Scalenut
User-agent: Scrap
User-agent: ScriptBook
User-agent: Seekr
User-agent: SEObot
User-agent: SEO Content Machine
User-agent: SEO Robot
User-agent: SemrushBot
User-agent: Sentibot
User-agent: Serper
User-agent: ShapBot
User-agent: Sidetrade
User-agent: Simplified AI
User-agent: Sitefinity
User-agent: Skydancer
User-agent: Skyvern
User-agent: SlickWrite
User-agent: SmartBot
User-agent: Sonic
User-agent: Sora
User-agent: Spider/2
User-agent: SpiderCreator
User-agent: Spin Rewrite
User-agent: Spinbot
User-agent: Stability
User-agent: Stable Diffusion
User-agent: StableDiffusionBot
User-agent: Stagehand
User-agent: Stealth
User-agent: Sudowrite
User-agent: SummalyBot
User-agent: Super Agent
User-agent: Superagent
User-agent: SuperAGI
User-agent: Surfer AI
User-agent: Taiko
User-agent: TavilyBot
User-agent: TerraCotta
User-agent: Text Blaze
User-agent: TextCortex
User-agent: Thinkbot
User-agent: Thordata
User-agent: TikTokSpider
User-agent: Timpibot
User-agent: Tinybird
User-agent: Together
User-agent: Traefik
User-agent: Tulu
User-agent: Tülu
User-agent: TurnitinBot
User-agent: TwinAgent
User-agent: uAgents
User-agent: UI-TARS
User-agent: VelenPublicWebCrawler
User-agent: Venus Chub AI
User-agent: Vidnami AI
User-agent: Vision RAG
User-agent: WARDBot
User-agent: WebSurfer
User-agent: WebText
User-agent: WebVoyager
User-agent: Webzio
User-agent: WeChat
User-agent: Whisper
User-agent: WordAI
User-agent: Wordtune
User-agent: WPBot
User-agent: Writecream
User-agent: WriterZen
User-agent: Writescope
User-agent: Writesonic
User-agent: WRTNBot
User-agent: xAI
User-agent: Xanthorox
User-agent: xBot
User-agent: XLNet
User-agent: YaK
User-agent: YaML
User-agent: YandexAdditional
User-agent: YouBot
User-agent: Zanista
User-agent: Zendesk
User-agent: Zero
User-agent: Zhipu
User-agent: Zhuque AI
User-agent: Zimm
Disallow: /
Important: Whenever making changes to your robots.txt file, take a few moments to validate the rules using a free online robots checker.

Block AI Bots via Apache/.htaccess

To actually enforce the “Ultimate AI Block List”, you can add the following rules to your Apache configuration or main .htaccess file. Like many others, I’ve written extensively on Apache and .htaccess. So if you’re unfamiliar, there are plenty of great resources, including my book .htaccess made easy.

In a nutshell, you can add rules via Apache/.htaccess to customize the functionality of your website. For example, you can add directives that help control traffic, optimize caching, improve performance, and even block bad bots. And these rules operate at the server level. So while bots may ignore rules added via robots.txt, they can’t ignore rules added via Apache/.htaccess (unless they falsify their user agent).

Using Apache? Check out my free, open-source 8G Firewall. 8G is lightweight, fast, and protects your site against a wide range of threats.

To block AI bots via Apache/.htaccess, add the following rules to either your server configuration file, or add to the main (public root) .htaccess file. Before making any changes, be on the safe side and make a backup of your files. Just in case something unexpected happens, you can easily roll back. That in mind, here are the Apache rules to block AI bots..

Blocks over 700+ AI bots and user agents.

Block list for Apache/.htaccess

Before using, read the Notes and Disclaimer.

# Ultimate AI Block List v1.8 20260105
# https://perishablepress.com/ultimate-ai-block-list/

<IfModule mod_rewrite.c>

	RewriteEngine On

	RewriteCond %{HTTP_USER_AGENT} (\.ai|-ai|_ai|ai\.|ai-|ai_|ai=|AddSearchBot|Agentic|AgentQL|Agent\ 3|Agent\ API|Agent-|AI\ Agent|AI\ Article\ Writer|AI\ Assistant|AI\ Chat|AI\ Content\ Detector|AI\ Detection|AI\ Dungeon) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (AI\ Journalist|AI\ Legion|AI\ RAG|AI\ Search|AI\ SEO\ Crawler|AI\ Training|AI\ Web|AI\ Writer|AI2|AIBot|aiHitBot|AIMatrix|Airesearch|AISearch|AITraining|Alexa|Alice\ Yandex|AliGenie|AliyunSec|Alpha\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (AlphaAI|Amazon|Amelia|AndersPinkBot|AndiBot|Anomura|Anonymous\ AI|Anthropic|AnyPicker|Anyword|Applebot|Aria\ AI|Aria\ Browse|Articoolo|Ask\ AI|Atlassian|AutoGen|Automated\ Writer|AutoML|Autonomous\ RAG) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (AutoRAG|Awario|AWS\ Trainium|Azure|BabyAGI|BabyCatAGI|BardBot|Basic\ RAG|Bedrock|Big\ Sur|Bigsur|Botsonic|Brightbot|Browser\ MCP\ Agent|Browser\ Use|BuddyBot|Bytebot|ByteDance|Bytespider|CarynAI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (CatBoost|CC-Crawler|CCBot|Chai|Channel3Bot|Character|Charstar\ AI|Chatbot|Chatsonic|ChatUser|Chinchilla|Claude|ClearScope|Clearview|Cognitive\ AI|Cohere|Common\ Crawl|CommonCrawl|Compass|Content\ Harmony) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Content\ King|Content\ Optimizer|Content\ Samurai|ContentAtScale|ContentBot|Contentedge|ContentShake|Conversion\ AI|Copilot|CopyAI|Copymatic|Copyscape|CoreWeave|Corrective\ RAG|Cotoyogi|CRAB|Crawl4AI|CrawlQ\ AI|Crawlspace|Crew\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (CrewAI|Crushon\ AI|Cypex|DALL-E|DarkBard|DarkBERT|DataFor|DataProvider|Datenbank\ Crawler|DeepAI|Deep\ AI|DeepL|DeepMind|Deep\ Research|DeepResearch|DeepSeek|Devin|Diffbot|Dolma|Doubao\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (DuckAssistBot|DuckDuckGo\ Chat|DuckDuckGo-Enhanced|Echobot|Echobox|Elixir|FacebookBot|FacebookExternalHit|Factset|Falcon|FIRE-1|Firebase|Firecrawl|Flyriver|Frase\ AI|FriendlyCrawler|Gato|Gemini|Gemma|Gen\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (GenAI|Generative|Genspark|Gentoo-chat|Ghostwriter|GigaChat|GLM|GodMode|Goose|GPT|Grammarly|Grendizer|Grok|Groq|GT\ Bot|GTBot|GTP|Hemingway\ Editor|Hetzner|Hugging) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Hunyuan|Hybrid\ Search\ RAG|Hypotenuse\ AI|iAsk|IbouBot|ICC-Crawler|Ideogram|ImageGen|Imagen|ImagesiftBot|ImageSpider|img2dataset|imgproxy|INK\ Editor|INKforall|Instructor|IntelliSeek|Inferkit|ISSCyberRiskCrawler|Janitor\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Jasper|Jenni\ AI|Julius\ AI|Kafkai|Kaggle|Kangaroo|Keyword\ Density\ AI|Kimi|Klaviyo|Knowledge|KomoBot|Kruti|KunatoCrawler|LAIONDownloader|LangChain|Langfuse|Language\ AI|Le\ Chat|Lensa|Leonardo) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Lightpanda|LightRAG|LinerBot|LinkupBot|LLaMA|LCC|LLM|Local\ RAG\ Agent|Lovable|Magistral|magpie-crawler|MAI-Image|Manus|MarketMuse|MBZUAI|Meltwater|Meta-AI|Meta-External|Meta-Webindexer|Meta\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (MetaAI|MetaTagBot|Middleware|Midjourney|Mini\ AGI|MiniMax|Mintlify|Mistral|Mixtral|model-training|Molmo|Monica|Nano|Narrative|NeevaBot|netEstate|Neural\ Text|NeuralSEO|NinjaAI|NodeZero) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (NotebookLM|Nova\ Act|NovaAct|Nytheon|OAI-SearchBot|OAI\ SearchBot|OASIS|Olivia|OLM|Omgili|Onyx|Open\ AI|Open\ Interpreter|OpenAGI|OpenAI|OpenBot|OpenInterpreter|OpenPi|OpenRouter|OpenText\ AI) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Operator|Outwrite|OVHcloud|Owlin|Page\ Analyzer\ AI|PaLM\ 2|PanguBot|Panscient|Paperlibot|Paraphraser\.io|peer39_crawler|Perflexity|Perplexity|Petal|Phind|PiplBot|Pixmo|Playwright|PoeBot|PoeSearchBot) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Poggio|Poseidon|ProWritingAid|Proximic|PulsarRPA|Puppeteer|Python\ AI|Python\ lxml|PyTorch|Qualified|Quark|QuillBot|Qopywriter|Qwen|RAG\ Agent|RAG\ Azure\ AI|RAG\ Chatbot|RAG\ Database|RAG\ IS|RAG\ Pipeline) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (RAG\ Search|RAG\ with|RAG-|RAG_|Raptor|React\ Agent|Reasoning|Redis\ AI\ RAG|Reka\ Flash|Replicate|RobotSpider|RunPod|Rytr|SaplingAI|SBIntuitionsBot|Scala|Scalenut|Scrap|ScriptBook|Seekr) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (SEObot|SEO\ Content\ Machine|SEO\ Robot|SemrushBot|Sentibot|Serper|ShapBot|Sidetrade|Simplified\ AI|Sitefinity|Skydancer|Skyvern|SlickWrite|SmartBot|Sonic|Sora|Spider/2|SpiderCreator|Spin\ Rewrite|Spinbot) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Stability|Stable\ Diffusion|StableDiffusionBot|Stagehand|Stealth|Sudowrite|SummalyBot|Super\ Agent|Superagent|SuperAGI|Surfer\ AI|Taiko|TavilyBot|TerraCotta|Text\ Blaze|TextCortex|Thinkbot|Thordata|TikTokSpider|Timpibot) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Tinybird|Together|Traefik|Tulu|Tülu|TurnitinBot|TwinAgent|uAgents|UI-TARS|VelenPublicWebCrawler|Venus\ Chub\ AI|Vidnami\ AI|Vision\ RAG|WARDBot|WebSurfer|WebText|WebVoyager|Webzio|WeChat|Whisper) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (WordAI|Wordtune|WPBot|Writecream|WriterZen|Writescope|Writesonic|WRTNBot|xAI|Xanthorox|xBot|XLNet|YaK|YaML|YandexAdditional|YouBot|Zanista|Zendesk|Zero|Zhipu) [NC,OR]
	RewriteCond %{HTTP_USER_AGENT} (Zhuque\ AI|Zimm) [NC]

	RewriteRule (.*) - [F,L]

</IfModule>
Important: Remember to test well before going live. You can use a free user-agent request tool to make requests posing as various AI bots.

Block AI Bots via Nginx

As with the previous Apache rules, you can go beyond robots.txt and actually enforce the “Ultimate AI Block List”, by adding the following rules to your Nginx configuration file. Like Apache, Nginx directives operate at the server level. So while bots may ignore rules added via robots.txt, they can’t ignore rules added via Nginx configuration (unless they falsify their user agent).

To block AI bots via Nginx, add the following rules to your main server configuration file. Before making any changes, be on the safe side and make a backup of your files. Just in case something unexpected happens, you can easily roll back. That in mind, here are the Nginx-formatted rules to block AI bots..

Blocks over 700+ AI bots and user agents.

Block list for Nginx

Before using, read the Notes and Disclaimer.

# Ultimate AI Block List v1.8 20260105
# https://perishablepress.com/ultimate-ai-block-list/

if ($http_user_agent ~* "(\.ai|-ai|_ai|ai\.|ai-|ai_|ai=|AddSearchBot|Agentic|AgentQL|Agent\ 3|Agent\ API|Agent-|AI\ Agent|AI\ Article\ Writer|AI\ Assistant|AI\ Chat|AI\ Content\ Detector|AI\ Detection|AI\ Dungeon)") {
	return 403;
}
if ($http_user_agent ~* "(AI\ Journalist|AI\ Legion|AI\ RAG|AI\ Search|AI\ SEO\ Crawler|AI\ Training|AI\ Web|AI\ Writer|AI2|AIBot|aiHitBot|AIMatrix|Airesearch|AISearch|AITraining|Alexa|Alice\ Yandex|AliGenie|AliyunSec|Alpha\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(AlphaAI|Amazon|Amelia|AndersPinkBot|AndiBot|Anomura|Anonymous\ AI|Anthropic|AnyPicker|Anyword|Applebot|Aria\ AI|Aria\ Browse|Articoolo|Ask\ AI|Atlassian|AutoGen|Automated\ Writer|AutoML|Autonomous\ RAG)") {
	return 403;
}
if ($http_user_agent ~* "(AutoRAG|Awario|AWS\ Trainium|Azure|BabyAGI|BabyCatAGI|BardBot|Basic\ RAG|Bedrock|Big\ Sur|Bigsur|Botsonic|Brightbot|Browser\ MCP\ Agent|Browser\ Use|BuddyBot|Bytebot|ByteDance|Bytespider|CarynAI)") {
	return 403;
}
if ($http_user_agent ~* "(CatBoost|CC-Crawler|CCBot|Chai|Channel3Bot|Character|Charstar\ AI|Chatbot|Chatsonic|ChatUser|Chinchilla|Claude|ClearScope|Clearview|Cognitive\ AI|Cohere|Common\ Crawl|CommonCrawl|Compass|Content\ Harmony)") {
	return 403;
}
if ($http_user_agent ~* "(Content\ King|Content\ Optimizer|Content\ Samurai|ContentAtScale|ContentBot|Contentedge|ContentShake|Conversion\ AI|Copilot|CopyAI|Copymatic|Copyscape|CoreWeave|Corrective\ RAG|Cotoyogi|CRAB|Crawl4AI|CrawlQ\ AI|Crawlspace|Crew\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(CrewAI|Crushon\ AI|Cypex|DALL-E|DarkBard|DarkBERT|DataFor|DataProvider|Datenbank\ Crawler|DeepAI|Deep\ AI|DeepL|DeepMind|Deep\ Research|DeepResearch|DeepSeek|Devin|Diffbot|Dolma|Doubao\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(DuckAssistBot|DuckDuckGo\ Chat|DuckDuckGo-Enhanced|Echobot|Echobox|Elixir|FacebookBot|FacebookExternalHit|Factset|Falcon|FIRE-1|Firebase|Firecrawl|Flyriver|Frase\ AI|FriendlyCrawler|Gato|Gemini|Gemma|Gen\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(GenAI|Generative|Genspark|Gentoo-chat|Ghostwriter|GigaChat|GLM|GodMode|Goose|GPT|Grammarly|Grendizer|Grok|Groq|GT\ Bot|GTBot|GTP|Hemingway\ Editor|Hetzner|Hugging)") {
	return 403;
}
if ($http_user_agent ~* "(Hunyuan|Hybrid\ Search\ RAG|Hypotenuse\ AI|iAsk|IbouBot|ICC-Crawler|Ideogram|ImageGen|Imagen|ImagesiftBot|ImageSpider|img2dataset|imgproxy|INK\ Editor|INKforall|Instructor|IntelliSeek|Inferkit|ISSCyberRiskCrawler|Janitor\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(Jasper|Jenni\ AI|Julius\ AI|Kafkai|Kaggle|Kangaroo|Keyword\ Density\ AI|Kimi|Klaviyo|Knowledge|KomoBot|Kruti|KunatoCrawler|LAIONDownloader|LangChain|Langfuse|Language\ AI|Le\ Chat|Lensa|Leonardo)") {
	return 403;
}
if ($http_user_agent ~* "(Lightpanda|LightRAG|LinerBot|LinkupBot|LLaMA|LCC|LLM|Local\ RAG\ Agent|Lovable|Magistral|magpie-crawler|MAI-Image|Manus|MarketMuse|MBZUAI|Meltwater|Meta-AI|Meta-External|Meta-Webindexer|Meta\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(MetaAI|MetaTagBot|Middleware|Midjourney|Mini\ AGI|MiniMax|Mintlify|Mistral|Mixtral|model-training|Molmo|Monica|Nano|Narrative|NeevaBot|netEstate|Neural\ Text|NeuralSEO|NinjaAI|NodeZero)") {
	return 403;
}
if ($http_user_agent ~* "(NotebookLM|Nova\ Act|NovaAct|Nytheon|OAI-SearchBot|OAI\ SearchBot|OASIS|Olivia|OLM|Omgili|Onyx|Open\ AI|Open\ Interpreter|OpenAGI|OpenAI|OpenBot|OpenInterpreter|OpenPi|OpenRouter|OpenText\ AI)") {
	return 403;
}
if ($http_user_agent ~* "(Operator|Outwrite|OVHcloud|Owlin|Page\ Analyzer\ AI|PaLM\ 2|PanguBot|Panscient|Paperlibot|Paraphraser\.io|peer39_crawler|Perflexity|Perplexity|Petal|Phind|PiplBot|Pixmo|Playwright|PoeBot|PoeSearchBot)") {
	return 403;
}
if ($http_user_agent ~* "(Poggio|Poseidon|ProWritingAid|Proximic|PulsarRPA|Puppeteer|Python\ AI|Python\ lxml|PyTorch|Qualified|Quark|QuillBot|Qopywriter|Qwen|RAG\ Agent|RAG\ Azure\ AI|RAG\ Chatbot|RAG\ Database|RAG\ IS|RAG\ Pipeline)") {
	return 403;
}
if ($http_user_agent ~* "(RAG\ Search|RAG\ with|RAG-|RAG_|Raptor|React\ Agent|Reasoning|Redis\ AI\ RAG|Reka\ Flash|Replicate|RobotSpider|RunPod|Rytr|SaplingAI|SBIntuitionsBot|Scala|Scalenut|Scrap|ScriptBook|Seekr)") {
	return 403;
}
if ($http_user_agent ~* "(SEObot|SEO\ Content\ Machine|SEO\ Robot|SemrushBot|Sentibot|Serper|ShapBot|Sidetrade|Simplified\ AI|Sitefinity|Skydancer|Skyvern|SlickWrite|SmartBot|Sonic|Sora|Spider/2|SpiderCreator|Spin\ Rewrite|Spinbot)") {
	return 403;
}
if ($http_user_agent ~* "(Stability|Stable\ Diffusion|StableDiffusionBot|Stagehand|Stealth|Sudowrite|SummalyBot|Super\ Agent|Superagent|SuperAGI|Surfer\ AI|Taiko|TavilyBot|TerraCotta|Text\ Blaze|TextCortex|Thinkbot|Thordata|TikTokSpider|Timpibot)") {
	return 403;
}
if ($http_user_agent ~* "(Tinybird|Together|Traefik|Tulu|Tülu|TurnitinBot|TwinAgent|uAgents|UI-TARS|VelenPublicWebCrawler|Venus\ Chub\ AI|Vidnami\ AI|Vision\ RAG|WARDBot|WebSurfer|WebText|WebVoyager|Webzio|WeChat|Whisper)") {
	return 403;
}
if ($http_user_agent ~* "(WordAI|Wordtune|WPBot|Writecream|WriterZen|Writescope|Writesonic|WRTNBot|xAI|Xanthorox|xBot|XLNet|YaK|YaML|YandexAdditional|YouBot|Zanista|Zendesk|Zero|Zhipu)") {
	return 403;
}
if ($http_user_agent ~* "(Zhuque\ AI|Zimm)") {
	return 403;
}
Important: Remember to test well before going live. You can use a free user-agent request tool to make requests posing as various AI bots.

Download plain-text list

Here is a plain-text version of the list. This list contains only the user-agent names, nothing else :)

Download Ultimate Block List for AI BotsVersion 1.8 ( 3.96 KB TXT )

Notes

Note: The four block lists above (robots.txt, Apache, Nginx, and Plain Text) are synchronized and include/block the same set of AI bots.

Note: Numerous user agents are omitted from the block lists because the names are matched in wild-card fashion. Here is a list showing wild-card blocked AI bots.

Note: The block lists focus on AI-related bots. Some of those bots are used by giant corporations like Apple, Amazon, and Facebook. So please keep this in mind and feel free to remove any bots that you think should be allowed access to your site. Also be sure to check the list of wild-card blocked AI bots.

Note: Each of the differently formatted block lists are case-insensitive. The robots.txt rules are case-insensitive by default, the Apache rules are case-insensitive due to the inclusion of the [NC] flag, and the Nginx rules are case-insensitive due to the tilde and asterisk ~*. So don’t worry about mixed-case bot names, their user agents will be blocked, whether uppercase, lowercase, or mIxeD cAsE.

Note: If you don’t care about search results, you can add the following rules to your robots.txt file. All search-related AI bots were removed from the block list in version 1.4 (see changelog below). Include the following only if you want to block major search engines like Bing, Google, and DuckDuckGo.

User-agent: Apple
User-agent: BingAI
User-agent: Bingbot-chat
User-agent: Bravebot
User-agent: Duck
User-agent: Google Bard AI
User-agent: Google-CloudVertexBot
User-agent: Google-Extended
User-agent: Google Gemini
User-agent: GoogleOther
User-agent: MSBot
Learn more: According to Google documentation, the value of the user-agent line (in robots.txt) is case-insensitive.

Changelog

Tip: The changelog below features the main changes like an overview, it does not chronicle every change to patterns, bots, details, and so forth. You can use a free online tool to compare block lists with previous versions, so you can figure out which bots have been added, removed, and other changes, etc.

v1.8 – 2026/01/05

  • Removes Flux
  • Consolidates GLM bots
  • Consolidates Awario bots
  • Adds 60+ new patterns, matching many more user agents
  • Streamlines numerous patterns, removes some patterns
  • Improves pattern matching for more accurate blocking and better performance
  • Updates the wild-card blocked AI bots plain-text list

v1.7 – 2025/09/24

  • Removes Apple, adds Applebot
  • Removes Brave Leo (Brave Browser Search)
  • Adds 65 new patterns, matching many more user agents
  • Streamlines numerous patterns, removes some patterns
  • Improves pattern matching for more accurate blocking and better performance
  • Updates the wild-card blocked AI bots plain-text list

v1.6 – 2025/07/18

  • Adds many new AI bots/patterns
  • Makes better use of wildcard matching to streamline the list while keeping false positives near zero
  • Adds robots.txt rules to allow all other bots full access
  • Improves alphabetization and formatting
  • Replaces AI2Bot and AI2 Labs with AI2
  • Replaces all specific Amazon bot names with Amazon
  • Replaces Applebot with Apple
  • Replaces LLMs with LLM
  • Replaces WormsGTP with GTP
  • Replaces Zero GTP and Zerochat with Zero

v1.5 – 2025/06/03

  • Adds Applebot
  • Changes Phindbot to Phind
  • Changes AI Search Engine to AI Search
  • Adds: AI Chat, ai-proxy, aiHitBot, AndiBot, AutoGLM, AutoML, BabyAGI, Brightbot, chatbot, Factset, imgproxy, Lensa, Lightpanda, Manus, Monica, NovaAct, Puppeteer, Qualified, Qwen, SemrushBot, TikTokSpider, Traefik, VelenPublicWebCrawler, YaML

v1.4 – 2025/04/17

  • Removes Applebot
  • Removes all 2 Bing agents
  • Removes all 4 Google agents
  • Changes PerplexityBot to Perplexity
  • Adds: Azure, Falcon, Genspark, GLM, ImageGen, Knowledge, LLMs, Nova Act, Operator, Sitefinity, Sonic, Super Agent, Zhipu
  • To restore blocking of search-related AI bots, check the notes above for a list

Previous versions

  • Version 1.3 – 2025/03/10 – Adds more AI bots, refines list to make better use of wild-card pattern matching of user-agent names.
  • Version 1.2 – 2025/02/12 – Adds 73 AI bots (Thanks to Robert DeVore)
  • Version 1.1 – 2025/02/11 – Replaces REQUEST_URI with HTTP_USER_AGENT
  • Version 1.0 – 2025/02/11 – Initial release.

Disclaimer

The information shared on this page is provided “as-is”, with the intention of helping people protect their sites against AI bots. The two block lists (robots.txt and Apache/.htaccess) are open-source and free to use and modify without condition. By using either block list, you assume all risk and responsibility for anything that happens. So use wisely, test thoroughly, and enjoy the benefits of my work :)

Support my work

I spend countless hours digging through server logs, researching user agents, and compiling block lists to stop AI and other unwanted bots. I share my work freely with the hope that it will help make the Web a more secure place for everyone.

If you benefit from my work and want to show support, please make a donation or buy one of my books, such as .htaccess made easy. You’ll get a complete guide to .htaccess and a ton of awesome techniques for optimizing and securing your site.

Of course, tweets, likes, links, and shares also are super helpful and very much appreciated. Your generous support enables me to continue developing AI block lists and other awesome resources for the community. Thank you kindly :)

Show support! Donate via PayPal, Stripe, or your favorite digital coin »

References

Thanks to the following resources for sharing their work with identifying and blocking AI bots.

Feedback

Got more? Leave a comment below with your favorite AI bots to block. Or send privately via my contact form. Cheers! :)

ai apache blacklist bots robots
About the Author
Jeff Starr = Web Developer. Security Specialist. WordPress Buff.
USP Pro: Unlimited front-end forms for user-submitted posts and more.
← Next Post
Previous Post →

58 responses to “Ultimate Block List to Stop AI Bots”

  1. Jason Reed 2026/01/05 1:58 pm • Reply

    Is there a feed or such that I can subscribe to so I can be notified when there’s a change?

    • Jeff Starr 2026/01/05 4:37 pm • Post Author • Reply

      Not really a dedicated feed for it, but I always try to mention project updates in the Thoughts feed. It contains a lot of other rambling stuff, but easier to add to an RSS/feed reader.

  2. MuP 2026/01/06 12:11 am • Reply

    Hi Jeff.

    Found this possible badbot in Apache’s logfile very often.
    Answered by 403 because of using greedy regex in htaccess since the beginning.
    mydomain . de is of course not my domain.

    Would You please check if the botname could be added in the next release?
    Thanks.

    213.61.218.0 - - [06/Jan/2026:01:45:25 +0100] "GET / HTTP/1.1" 403 - mydomain.de "http://www.google.de/" "finbot" "-"

  3. Kristina Ponting 2026/01/11 11:41 pm • Reply

    It is a false Googlebot! On way to avoid it is: 

    RewriteCond %{REMOTE_ADDR} !^64\.249\.
RewriteCond %{HTTP_USER_AGENT} Googlebot RewriteRule .* - [F]

    or: 

    <IfModule mod_rewrite.c>
	RewriteCond %{HTTP_USER_AGENT} ^Mozilla/5\.0\ \(compatible;\ Googlebot/2\.[01];\ \+http://www\.google\.com/bot\.html\)$
	RewriteCond %{HTTP:Accept} ^\*/\*$
	RewriteCond %{HTTP:Accept-Encoding} ="gzip,deflate"
	RewriteCond %{HTTP:Accept-Language} =""
	RewriteCond %{HTTP:Accept-Charset}  =""
	RewriteCond %{HTTP:From} ="googlebot(at)googlebot.com"
	RewriteCond %{REMOTE_ADDR} ^66\.249\.(6[4-9]|7[0-9]|8[0-46-9]|9[0-5])\. [OR]
	RewriteCond %{REMOTE_ADDR} ^216\.239\.(3[2-9]|[45][0-9]|6[0-3])\.0
	RewriteCond %{REMOTE_HOST} ^crawl(-([1-9][0-9]?|1[0-9]{2}|2[0-4][0-9]|25[0-5])){4}\.googlebot\.com$
	RewriteRule ^ – [S=1]
	RewriteCond %{HTTP_USER_AGENT} Googlebot [NC]
	RewriteRule ^ – [F]
</IfModule>
  4. JD 2026/08/10 7:42 pm • Reply

    I’m at a loss on a cpanel server how your apache mod_rewrite statements to block bots will work when in the individual .htaccess file for each site. But to apply it server-wide, I would want to put it in the apache global configuration, Pre main or Pre Virtual host sections to apply to all sites (cpanel’s method of modifying the httpd configs). When I put the mod_rewrite statements in the cpanel either of the global configuration sections, they don’t work as tested with a browser UA switcher. Any one have troubleshooting steps or idea on how to get this to work?

  5. Kristina Ponting 2026/09/24 8:04 pm • Reply

    The core problem is that mod_rewrite configurations are not inherited by default. According to the official Apache mod_rewrite documentation, rules defined in the global configuration (like Pre Main or Pre VirtualHost) are completely ignored inside individual VirtualHost containers and .htaccess files unless you explicitly tell Apache to inherit them. To fix this and make your server-wide bot-blocking rules actually work, follow these three steps:1. Enable Global Rewrite Inheritance You need to add a directive that forces all websites (VirtualHosts) to inherit the rules you defined globally.Log into WHM as root.Go to Service Configuration -> Apache Configuration -> Include Editor. In the Pre Main Include or Pre VirtualHost Include section (wherever you placed your bot-blocking code), ensure you add this line right above or below your rules:apacheRewriteOptions InheritDownBefore

    Use the code with caution. (Note: Using InheritDownBefore ensures your global blocking rules execute before any individual site rules, stopping the bot immediately).2. Check for RewriteEngine OnEnsure you actually turned the engine on in that same global block: apacheRewriteEngine On
    RewriteOptions InheritDownBefore

    # Your bot blocking rules here
RewriteCond %{HTTP_USER_AGENT} (badbot1|badbot2|badbot3) [NC]
RewriteRule .* – [F,L]

    Use the code with caution.3. Rebuild and Restart ApachecPanel requires you to rebuild the configuration file for changes made via the Include Editor to stick:Scroll down and click Update.Click Rebuild Configuration and Restart Apache.If the rules still aren’t catching the User-Agent switcher after this change, let me know:What specific rewrite rules are you using to block the bots?Are your test domains running over HTTPS or behind a proxy like Cloudflare?I can help you tweak the syntax to ensure it triggers correctly!

• 123

Leave a reply

Name and email required. Email kept private. Basic markup allowed. Please wrap any small/single-line code snippets with <code> tags. Wrap any long/multi-line snippets with <pre><code> tags. For more info, check out the Comment Policy and Privacy Policy.

Subscribe to comments on this post

Welcome
Perishable Press is operated by Jeff Starr, a professional web developer and book author with two decades of experience. Here you will find posts about web development, WordPress, security, and more »
BBQ Pro: The fastest firewall to protect your WordPress.
Thoughts
Working on a complete redesign of Perishable Press :)
The Office is Cheers with more irony.
More fine-grained control of macOS screen brightness: hold down Option + Shift before pressing either of the brightness keys. You get 64 increments instead of the usual 16.
REST Pro Tools featured in David McCan’s new video on locking down WordPress.
Gonna try exercise.
Launching my new plugin, REST Pro Tools 🛠️ Granular control of the WP REST API.
The algorithm is way too hypersensitive.
+ More thoughts »
Newsletter
Get news, updates, deals & tips via email.
Email kept private. Easy unsubscribe anytime.
Topics
Perishable Press Newsletter
Get news, updates, deals & tips via email.
Your email will remain private. Easy unsubscribe anytime.
Find me on the social medias
Around the site
WordPress help
Favorite projects
© 2004–2026 Perishable Press • Yes Theme by Monzilla Media • Built w/ shapeSpace • Sitemap • Policy • RSS • John 3:16