Post Thumbnail

Cloudflare launched Pay per Crawl: AI bots will pay for scanning sites

Cloudflare company launched a very interesting service! It allows website owners to charge AI bots money for data collection! The service is called Pay per Crawl and is currently running in closed beta version.

It’s important to understand the scale. Cloudflare serves 20% of the entire internet. And through this platform, website owners will be able to choose. Whether to allow AI bots to scan their content for free, block them completely, or charge micropayments for each scan.

The service makes it clear what exactly bots are collecting information for. For training AI models, for appearing in chatbot responses, or for other purposes.

Major publications have already joined the initiative – Conde Nast, TIME, Associated Press and others. They will block AI bots by default.

Statistics from Cloudflare show a curious picture. Google search engine scans sites 14 times for each received transition. And OpenAI’s bot – 17,000 times for 1 transition, and Anthropic – 73,000 times!

Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.

Latest News

Historic milestone: Amazon's millionth robot delivered to Japan

Amazon reached a historic milestone! And after 13 years of implementing robots in its warehouse facilities, the company announced reaching the mark of 1 million robotic devices. The millionth robot was recently delivered to an Amazon warehouse in Japan.

Chinese New Year canceled: Alibaba works nights on DeepSeek response

A real technological race erupted in China, where Alibaba engineers faced an unprecedented situation. They had to cancel their vacations. And work even during Chinese New Year. And this is one of the most significant and revered holidays in all of China!

Perplexity releases browser with AI for automating online tasks

Perplexity company is preparing to release its own browser with integrated artificial intelligence. Startup CEO Aravind Srinivas announced that the release is already in its final stage. And users can submit applications for early access through the official website. Link in description.

Buterin criticized Altman's World project for threat to pseudonymity

Ethereum co-founder Vitalik Buterin expressed serious concerns about the World project, Sam Altman's brainchild! This is an extremely interesting technological dispute about the future of digital identification.

Amazon and Anthropic are building a supercomputer 5 times more powerful than the current one

Anthropic and Amazon are building something colossal! The companies have joined forces to create the Project Rainier supercomputer. The most powerful in Amazon Web Services history. It will be housed in multiple data centers across the United States.