Stanford researchers find Mastodon has a massive child abuse material problem

In my opinion the biggest issue the author points out is that cached materials are sometimes retained even after moderator action. Which honestly just sounds like a straight up bug more than anything. Though if I were running an instance, the feds showing up at my door with a warrant because I've been accidentally distributing CSAM would be my nightmare scenario. And of course jurisdiction plays a part, too: an American user on a Canadian server might see drawn depictions of sexualized minors, think "weird but not illegal," and now the Canadian admin has content that's illegal in Canada on their Canadian server and has no idea.

IMO I think the best solution to this is something similar to what Renaud Chaput (Mastodon's resident infra boffin) described in his recent blog post. Effectively, give admins a way to hand this off to pluggable third-party services. Admins that are worried about this sort of thing can then have some degree of safety via e.g. PhotoDNA, whereas others can take on additional risk and preserve additional privacy.

All that said: yeah the headline makes it sound like .social is some 8chan-esque hellhole, whereas in reality my feed is 99% German programmers sharing milquetoast political takes.

The person outright rejects defederation as a solution when it IS the solution, if an instance is in favor of this kind of thing you don't want to federate with them, period.

I also find worrying the amount of calls for a "Fediverse police" in that thread, scanning every image that gets uploaded to your instance with a 3rd party tool is an issue too, on one side you definitely don't want this kinda shit to even touch your servers and on the other you don't want anybody dictating that, say, anti-union or similar memes are marked, denounced and the person who made them marked, targeted and receiving a nice Pinkerton visit.

This is a complicated problem.

Edit: I see somebody suggested checking the observations against the common and well used Mastodon blocklists, to see if the shit is contained on defederated instances, and the author said this was something they wanted to check, so i hope there's a followup

That'd be useless though, because first, it'd probably opt-in via configuration settings and even if it wasn't, people would just fork and modify the code base or simply switch to another ActivityPub implementation.

We're not gonna fix society using tech unless we're all hooked up to some all knowing AI under government control.

Facebook actually being good for once? Unheard of

While white knights propose ways to control everyone everywhere everytime, in the name of catching the pedos who will just hop to the next platform (or have already).

It doesn't help to bring whataboutism into this discussion. This is a known problem with the open nature of federation. So is bigotry and hate speech. To address these problems, it's important to first acknowledge that they exist.

Also, since fed is still in the early stages, now is the time to experiment with mechanisms to control them. Saying that the problem is innate to networks is only sweeping it under the rug. At some point there will be a watershed event that'll force these conversations anyway.

The challenge is in moderating such content without being ham-fisted. I must admit I have absolutely no idea how, this is just my read of the situation.

This is exactly what I thought. The story here is that the human race has a massive child abuse material problem.

The internet itself is a network with major CSAM problems

Is it, though?

Over the last year, I've seen several reports on TV of IRL group abuse of children, by other children... which left everyone scratching their heads as to what to do since none of the perpetrators are legally imputable.

During that same time, I've seen exactly 0 (zero) instances of CSAM on the Internet.

Sounds to me like IRL has a major CSAM, and general sex abuse, problem.

You moved the period in the wrong direction. It's 0.034462%.

That's just the material they knew was CSAM from previous investigations.

There were also 713 uses of the top 20 CSAM-related hashtags across the Fediverse on posts that contained media, as well as 1,217 text-only posts that pointed to “off-site CSAM trading or grooming of minors.” The study notes that the open posting of CSAM is “disturbingly prevalent.”

What's the reason for that? Caching purposes?

I agree, but who’s going to pay for it?

How about police/the tax payer?

If university researchers can find the stuff, then police can find it too. There should be an established way to flag the user (or even the entire instance) so that content can be removed from the fediverse while simultaneously asking for all data that is available to try to catch the criminals.

And of course, if regular users come across anything illegal they will report it too, and it should be removed quickly (I'd hope immediately in many cases, especially if the post was by a brand new/untrusted account).

One way to do this is to block hashes. This is a slippery slope though because it could be used maliciously. Only way to do this and protect freedom of information is to make this fully open source.

The researchers can't be taken seriously if they don't acknowledge that you can't force free software to do something you don't want it to.

Even if we started way down at the stack and we added a CSAM hash scanner to the Linux kernel, people would just fork the kernel and use their own build without it.

Same goes for nginx or any other web server or web proxy. Same goes for Tor. Same goes for Mastodon or any other Fedi/ActivityPub implementation.

It. Does. Not*. Work.

Please, prove me wrong, I'm not all knowing, but short of total surveillance, I see no technical solution to this.

Probably just the ease at which you can find it since each instance is linked, it basically becomes a search engine that might not have the same controls/protection as Google etc

Privately hosted websites are only useful for established clients. Via social media and image sharing platforms the distributors try to reach new clientele. They often have more or less hidden tags and codes that can attract potential customers. When someone reacts to these they carefully try to see if the person could be trusted to have access to private sharing. It's how drug dealers online sometimes work or extremist groups.

Total tangent, but we kid ourselves if we think the fediverse is somehow censorship-immune in comparison to Reddit or Twitter.

There are more moderators and administrators across all instances which can federate/defederate at will and can delete posts and propagate this deletion through the network. At the same time governments don’t need to negotiate with a large company, but only need to hint they could destroy one person’s livelihood to remove undesirable content from the network. And to avoid the Streisand effect instead of requesting to delete one specific piece of subversive content (which could backfire), just insinuate some illegal material (CSAM being the most obvious, but anything goes, really) has been found to force shut down or takeover of the whole instance.

The same goes for big companies instead of governments: if a large corporation has launched their own Mastodon clone, the first thing they’d reasonably fund are smearpieces by “journalists” and/or “scientists” hinting at harm to befall server owners by continuing to host Mastodon instances.

I personally hate, what crypto has become (if I wanted to destroy crypto, I’d have invented crypto bros as a psy op), but the fediverse isn’t really federated enough to be resistant to influence by corporations and governments and something blockchain adjacent could have been the solution. For example: if the server admin and their hoster is totally unable to decrypt whatever is stored on their own server and the network as a whole is distributing all the content probabilistically across every federated server, the network would only get stronger and more censorship resistant with each new instance. If the government is forcing you for any reason to take down your server your content is not gone but stored with all the other nodes. If you are able to retrieve your key, you could even move to a new instance and authenticate as your old instance (don’t forget: you are not “sending” BTC from one wallet to another, you are only telling as much nodes as sensible that BTC on the chain belongs to a new key now; the same would go for content. Take down one node with a “wallet” doesn’t change which wallet the BTC on the chain belongs to. I propose the same, just with content). If federation between instances would work in a comparable way as it is now, this would additionally increase the probability to root out bad faith actors trying to flood the whole network with illegal content, since their content would be stored on much less nodes in a pseudo-predictable way: as soon as each major instance would defederate, their content would not be stored on their nodes and unfederated third-party-nodes.