AI Crawl Control is now in the dashboard, in beta: opt-in per site and starting in log mode. It classifies every request against a catalog of 31 AI bots from 18 operators, verified by published IP ranges, reverse DNS and Web Bot Auth signatures, flags crawlers that only claim to be someone, and lets you allow, log, challenge, block or tarpit per category or per bot. It also manages robots.txt with Content-Signal, serves Markdown to agents, protects your own AI endpoints and scores how ready a site is for AI agents. The MCP tools ai_agents_report, ai_agents_settings, ai_agents_catalog and agent_readiness_scan cover the same ground.
See the AI Crawl Control documentation.