iFixit reports that Anthropic's ClaudeBot search bot visited the company's website almost a million times in 24 hours, violating the repair company's Terms of Service. This was reported by The Verge.
Kyle Wiens, CEO of iFixit, wrote to Anthropic in X. He writes that the company recorded a visit to the iFixit website by a bot from Anthropic. After that, Wiens published screenshots of his communication with Claude, where the chatbot itself reports that it is forbidden to use content from the iFixit website for training.
If any of those requests accessed our terms of service, they would have told you that use of our content expressly forbidden. But don't ask me, ask Claude!
— Kyle Wiens (@kwiens) July 24, 2024
If you want to have a conversation about licensing our content for commercial use, we're right here. pic.twitter.com/CAkOQDnLjD
"You're not only taking our content without paying, you're tying up our devops resources. If you want to have a conversation about licensing our content for commercial use, we're right here," writes Kyle Wiens.
The iFixit Terms of Use state that "reproduction, copying, or distribution" of any content from the website "is strictly prohibited without the prior written permission" of the company, including "training a machine learning or artificial intelligence model."
Anthropic responded to 404media by referring to its FAQ page, which says that its search engine can be blocked. To do this, you need to use the robots.txt file.
Kyle Wiens confirmed that after they added the crawl-delay extension to their robots.txt, any ClaudeBot visits to the site stopped.
OpenAI uses the same format of banning. But, for example, Perplexity, which develops Perplexity.ai, completely ignores the rules in robots.txt.