Reddit’s is taking a stand against AI crawlers | TechCrunch Minute

Reddit is taking a stand against AI Companies or at least asking them to Cough up some money earlier this week Reddit announced a change to its robots Exclusion protocol also known as its Robots. text file I know that probably Sounds kind of boring but it's part of This larger maybe battle maybe Negotiation between the AI companies Which are hungry for content to feed Their models and the companies that Control that content actually if you've Been following the news around AI you've Probably heard the phrase robots. text Fairly recently it's how websites Communicate to third parties about how They can be crawled the classic example Is websites that allow Google to crawl Them so they can be included in search Results obviously most websites do that It's a pretty straightforward value Exchange though even there there's some Dispute about whether Google provides Enough value especially to news Organizations when it comes to AI that Value exchange is a lot less obvious if I run a website whose business model Depends on clicks and eyeballs why would I allow an AI company to Hoover up my Content and then not send me traffic and In some cases outright plagiarize me That's essentially how the Forbes team React when they discovered that Perplexity AI was being used to create

Pages that were just full text versions Of Forbes articles then wied published An investigation showing that perplexity Was ignoring robots. text files to not Scrape its website and that brings us Back to Reddit by changing its robots. Text file and also by rate limiting and Blocking unknown Bots and crawlers Reddit is presumably trying to prevent The behavior that perplexity was Criticized for to be clear Reddit isn't Opposed to working with AI companies It's already announced Partnerships with Open Ai and Google it just wants to be Selective about who it partners with Plus it probably wants to be paid for Access to all its content we're Definitely going to be hearing more About this the CEO of perplexity has Defended himself by saying that robots. Text files are not legal Frameworks so It's possible that AI companies will Just ignore reddit's wishes and try to Find technical workarounds in fact the Relationship between Ai and the web Feels pretty wild west right now which Would be more fun and exciting if there Wasn't the possibility that entire Industries were about to get killed off There are companies fighting over the Legal guardrails around Ai and content For example the New York Times is suing Open Ai and Microsoft I suppose the AI Companies might defend their ask of

Forgiveness not permission approach by Saying they represent the future so the Media companies that represent the past Need to get on board but the truth is Without content Partners like Reddit the Development of AI models might slow or Even stall so they might not have much Of a future at all I'll see you tomorrow


Coinbase is a popular cryptocurrency exchange. It makes it easy to buy, sell, and exchange cryptocurrencies like Bitcoin. Coinbase also has a brokerage service that makes it easy to buy Bitcoin as easily as buying stocks through an online broker. However, Coinbase can be expensive due to the fees it charges and its poor customer service.

Leave a Comment

    • bitcoinBitcoin (BTC) $ 65,312.00 0.97%
    • ethereumEthereum (ETH) $ 3,329.35 4.56%
    • tetherTether (USDT) $ 0.999767 0.04%
    • bnbBNB (BNB) $ 572.39 1.78%
    • solanaSolana (SOL) $ 177.02 2.51%
    • xrpXRP (XRP) $ 0.615028 2.92%
    • usd-coinUSDC (USDC) $ 0.999851 0.07%
    • staked-etherLido Staked Ether (STETH) $ 3,328.98 4.53%
    • dogecoinDogecoin (DOGE) $ 0.127488 2.31%
    • the-open-networkToncoin (TON) $ 6.85 0.23%