New research: The Binary Switch
AI agents now do things on the web that cost money. They book, they buy, they cancel, they fill in forms on a person's behalf. So we asked a plain question: when an agent shows up at a website, how does that site decide what it is allowed to do?
We measured it, across the top 500 sites by inbound links and a hand-picked set of transactional sites (booking, travel, retail, ticketing, food delivery). Three measurements, one measurement tool, all the raw data published. The paper is The Binary Switch: How Websites Govern Autonomous Agent Access.
What we found
Nobody says what an agent may do. Every site we could reach governs reading: which pages a crawler may fetch, and what may be done with the text afterwards. Not one site in the sample says whether an agent may buy something, book something, or cancel something. Not a majority. Zero.
The published rules are not the enforced rules. We knocked on each site as five different identities. On transactional sites, the robots.txt file welcomed an AI agent that the server then turned away about a fifth of the time, and not one site with a "keep out" rule in its file actually enforced it. The document and the thing that decides are two unrelated systems.
Sites already have the vocabulary for a dial. They just do not point it at AI. The same operator who writes a dozen precise rules for Google's crawler writes a single line for an AI one: keep out entirely. They fence exactly the things you would expect them to worry about, checkouts, seats, pricing, at well above the rate of the web at large. Then, for AI, they discard the nuance.
Why that is the interesting part
The obvious reading is that sites lack the words for a graded policy, so someone should invent them. Our measurements say that reading is wrong. Operators have the words and use them daily. They fall back to an all-or-nothing switch because a rule that nothing enforces, aimed at a party that cannot be held to account, is worth nothing.
There is also a deeper objection the paper takes on. Graded access looks impossible because intent is invisible: an agent booking a room and an agent quietly copying your prices send identical requests. We argue that this framing is a mistake. Every one of those risks is defeated by counting against an accountable identity, not by guessing at motive. Holding every middle seat on a flight to force the sale of the aisle and window seats only works while the two sessions are anonymous and unlinked. Tie them to one verified party and "six seats held, nothing bought" is simple arithmetic. The right unit of policy is the party, not the request.
We are also candid about the work's limits: two measurement bugs we found and fixed mid-study, an analysis error we corrected, and the reasons our numbers are a floor rather than a ceiling. That section is in the paper, not hidden in a footnote.
Read it, argue with it, or ask us for the raw data. It is here, with the PDF and the full source.