Briefing

Google Introduces Google‑Agent User Agent for AI Browsing

security
by Slobodan Manic · Claude

Update your robots.txt and authentication to allow Google‑Agent while protecting sensitive pages.

What to do now

Add Google‑Agent to your allowed list and implement web‑bot‑auth if needed.

Summary

On March 20, 2026 Google added a new user agent string, Google‑Agent, to its official list of web fetchers, specifically for AI systems running on Google infrastructure that browse websites on behalf of users. Google‑Agent is a user‑triggered fetcher that ignores robots.txt rules, unlike other bots such as GPTBot or Claude‑User, which respect robots.txt. The new agent is used by Project Mariner, Google’s experimental AI browsing tool, and is classified as a web‑bot‑auth protocol experiment using the identity https://agent.bot.goog, which provides cryptographic signatures for bot requests. Website owners must now consider server‑side authentication or access controls to restrict Google‑Agent, as robots.txt alone is insufficient. Google publishes IP ranges for Google‑Agent, and sites should monitor logs for the user agent string containing “compatible; Google‑Agent” to track agent activity. Forms and multi‑step processes may fail if they rely on JavaScript patterns that confuse automated systems, so semantic HTML and clear labels remain essential. The article advises updating firewall rules, ensuring Google‑Agent is permitted, and implementing web‑bot‑auth if needed to secure sensitive content. This change marks a shift from crawler‑centric access control to a three‑tier visitor model: humans, crawlers, and real‑time agents.

Key changes

  • Google‑Agent ignores robots.txt and is user‑triggered
  • Uses web‑bot‑auth with identity https://agent.bot.goog for cryptographic signatures
  • Google publishes IP ranges for verification
  • Sites must monitor logs for "compatible; Google‑Agent" user agent
  • Forms may fail if they rely on JavaScript patterns that confuse automated systems

Affects

internal

Customer impact

Analyzing matches…

Ask about this story

Impact on an agency? Which customers? Compare historically Risks of waiting