Cloudflare has introduced a beta Web Search API through AI Gateway. The API is available via REST and Cloudflare Workers, with unified logging, access control, and billing, plus support for bringing your own provider keys.
It lets AI agents retrieve real-time web search results from Ceramic.ai, Exa, and Linkup. A single interface ties search billing into AI Gateway so search requests and model inference can be logged together. Providers must follow verified-bot requirements and return source links.
The API allows AI applications to pull current web content—including recent news, API changes, and technical documentation—and pass those results to a language model as context during inference. Cloudflare group product manager Michelle Chen, senior director Sam Else, and senior systems engineer Gabriel Massadas described the issue the company is targeting:
When you use an agent that needs to fetch a page in real time, the agent usually just guesses the URL and then calls a tool to fetch it with curl. That is why web fetches sometimes return a 404 Not Found—typically because the agent guessed the wrong URL. Randomly guessing URLs is not an efficient approach.
Applications and external services reach web search through a REST endpoint by supplying an AI Gateway authentication token and naming the search provider in the request payload. Chen, Else, and Massadas added:
Crawlers used by web search providers must meet the public “verified bots” requirements defined in Cloudflare’s developer documentation, and search responses must include links to the crawled content. These rules support a fairer internet and let content creators decide how their data is handled.
On a popular Hacker News thread, commenters discussed search quality, data-retention limits, and whether the unified API is a meaningful improvement over existing options. Simon Willison wrote:
For a search API, my first question is whether it lets you store and republish the results. If I am running an agent system and cannot store responses—or offer a “share conversation” button—that is a significant limitation. The answer is often buried in the terms of service.
Maximilian Goisser, CTO and co-founder of Erato Labs, said:
For companies already using Cloudflare, it is gradually becoming a trusted primary cloud provider (a role usually held by AWS, GCP, or Azure). Most of the company’s cloud spend would be settled on that platform, so there would be no need to reopen procurement.
Cloudflare said it will select providers that support zero data retention and will offer search at each provider’s list price with no added fee. It plans to add native server-side tools to AI Gateway, starting with web search. Until then, developers can integrate web search into workflows with Cloudflare Workers.
Original article: https://www.infoq.cn/article/SgXuVvzbHr8lKrihTAZW