“We believe in an open internet… as long as you use these specific services.”
This really sucks. So we’re looking at a future where search engines are like streaming services now. “Hmmm now which search engine was <insert website here> on?”
I don’t have a ton of knowledge in this area, but this seems like it should run afoul of antitrust regulations?
Who should be regulated, Google or Reddit? Reddit updated there robots.txt to disallow everything. As it’s their site, I guess it’s also their right to determine that. They then made a deal with Google, which I guess is also not abusing a dominant position by Google, as Reddit could have made a deal with anyone.
Yeah but reddit made a deal with google because google’s the big player.
It’s hard to say, but I’d lead toward Google on this one. How does reddit benefit from only being indexed by one search engine? Google must have offered them something more, to make it in reddit’s best interests.
In other words, this deal naturally benefits only google, at the cost of value to reddit and to the public. So google must be doing something that makes it worth it to reddit. Could be threat of punishment: “You give us exclusive crawl access, or we don’t crawl you”.
In 2023, Reddit decided to start charging exorbitant amounts for API access, making it non-viable for free 3rd party apps to access its content, citing things like AI crawlers “stealing” their (users’) content.
In 2024, Google announced an agreement with Reddit to access the API, citing things like enhanced up to date search results. I don’t recall having seen whether they pay for it, or how much, but possibly they do.
It would stand to reason, that if Reddit has managed to get a single dime for API access, and they keep thinking free access to their users’ content is “stealing”, then Reddit would be interested in making it as hard as possible to access the content without paying.
Could be threat of punishment: “You give us exclusive crawl access, or we don’t crawl you”.
That could’ve been part of the agreement: “You give us cheap/free API access, or we don’t crawl you”.
Reddit tightening things down while trying to sell API access, just happens to benefit Google.
That was my first thought too. Yet another reason to vote for Dems this November - only one party actually gives a shit about enforcing antitrust regulations!
are you absolutely positive the democrats give a shit about antitrust regulations? Biden did actively strike break.
He did at the beginning, but he helped them get what they wanted in the end, and I think that counts for something.
“We’re thankful that the Biden administration played the long game on sick days and stuck with us for months after Congress imposed our updated national agreement,” Russo said. “Without making a big show of it, Joe Biden and members of his administration in the Transportation and Labor departments have been working continuously to get guaranteed paid sick days for all railroad workers.
“We know that many of our members weren’t happy with our original agreement,” Russo said, “but through it all, we had faith that our friends in the White House and Congress would keep up the pressure on our railroad employers to get us the sick day benefits we deserve. Until we negotiated these new individual agreements with these carriers, an IBEW member who called out sick was not compensated.”
Oh yeah you’re right we should just not even bother voting and let the right wing win.
It’s a bit of a dilemma reading their policy:
We believe in the open internet and in keeping Reddit publicly accessible to foster human learning (…) Unfortunately, we see more and more entities using unauthorized access (…) especially with the rise of use cases like generative AI. This sort of misuse of public data has become more prominent as more and more platforms close themselves off from the open internet.
We still believe in an open internet, but we do not believe that third parties have a right to misuse public content just because it’s public.
Being a open/public platform, but still wanting to protect user’s content from being used for AI could be a good thing, and I guess also what many fediverse users would want for this platform. Making a distinction between AI and search indexing could indeed be difficult. But then making content deals with Google for search indexing and AI training is a bit hypocrite.
Ummmmmmmm. This seems illegal. Is this not illegal?
Does this mean the Internet Archive will no longer be archiving reddit posts? That’s how I’ve tried viewing most since I deleted my accounts.
I honestly do not think Internet Archive even should be archiving such behemoths like Reddit or Twitter. Only thing it should keep would be currently dead sites.
Even worse when people are accessing these posts through Archive even when there is a live copy. A lot of storage and bandwidth wasted.
…but at some point those giant sites may go offline. I see the point of archiving them now for posterity, but you’re right. The archive shouldn’t be used as a concurrent mirror of those sites for privacy reasons.
I have my browser set up to redirect Reddit links to libreddit instances for that purpose.
But imagine this… an immoral rich human being, who’s family got rich by mining blood rubies in south Africa, buys reddit for 50B$. This person fires half the people and refuses to pay the bills for servers and the servers shut down… how will you access your favorite GoneWild posts? This is all fictional of course.
Archives are ideal for identifying sneaky behavior like that. You never know when an admin might have the ability to delete or edit something without anyone noticing.
