- cross-posted to:
- technology@lemmy.world
- technews@radiation.party
- cross-posted to:
- technology@lemmy.world
- technews@radiation.party
The New York Times blocks OpenAI’s web crawler::The New York Times has officially blocked GPTBot, OpenAI’s web crawler. The outlet’s robot.txt page specifically disallows GPTBot, preventing OpenAI from scraping content from its website to train AI models.
You must log in or register to comment.
as if a text file is going to stop them
deleted by creator
NYT also uses a third party bot identification and mitigation service.
The question is: Does that crawler adhere to robot.txt policies?
They made a flag specifically for their crawler, so they can say that they do but in the most annoying way possible.
removed by mod
what is the ai being trained for anyways, how to be a NYT journalist?
