The company wants developers to stop straining its website, so it created a cache of Wikipedia pages formatted specifically for developers.

Data science platform Kaggle is hosting a Wikipedia dataset that’s specifically optimized for machine learning applications.

Wikipedia.org, for example, doesn't bother to block AI crawlers from Google, OpenAI, or Anthropic in its robots.txt file. It blocks a number of bots deemed troublesome for their penchant for slurping ...