How can the Fediverse protect against AI slop?

Reminds me of this one:

source

There are two groups here, bots, and bad actors. We've found that these measures have mostly stopped them both.

Bots

• Registration applications. Its been extremely easy to differentiate bots from real people by asking a series of simple questions, and only let the real people in.
• Reports: so that mods / admins can see them quickly.
• Blocking open-signup servers that don't have required applications, that usually serve as spam-attacks against the whole fediverse.

Some bots still get through occasionally, but not many compared to before. And some servers have more "lax" application questions, so they let more through.

Bad actors

• Registration applications. Most of the trolls are of a temperament where they refuse to do the work of answering questions earnestly. They can't help themselves but give obviously trolling answers, if they do even bother to do that work at all.
• Reports: same as above.
• Ban + remove. Mods and admins can ban and remove all a person's content at the click of a button. So even if the troll did the work of getting past the front door, then all their work is nullified by an action that takes less than 5 seconds. So they wasted much more of their time, than they did for admins, and accomplished nothing lasting.

Hi there! Admin of Tucson.social here.

I think that the only way the fediverse can honestly handle this is through local/regional nodes not interest based global nodes.

Ideally this would manifest as some sort of non-profit entity that would work with municipalities to create community owned spaces that have paid moderation.

So then comes the problem of folks not agreeing with a local nodes moderation staff - but that's also WHY it should be local. It's much easier to petition and organize against someone who exists in your town than some guy across the globe who happens to own a large fediverse node.

This model just doesn't work (IMO) if nodes can't be accountable to a local community. If you don't like how Mastodon, or lemmy.world are moderated you have zero recourse. For Tucson.social - citizens of Tucson can appeal to me directly, and because they are my fellow citizens I take them FAR more seriously.

Only then will people be trusting enough to allow for the key element to protecting against AI Slop. Human Indemnification Systems. Right now, if you wanted to ask the community of lemmy.world to provide proof they are human, you'd wind up with an exodus. There's just no trust for something like that and it would be hard to acquire enough trust.

With a local node, that conversation is still difficult, but we can do things that just don't scale with global nodes. Things like validating a person by meeting them to mark them as "indemnified" on a platform, or utilizing local political parties to validate if a given person is "real" or not using voter rolls.

But yeah, this is a bit rambly, but I'll conclude that this is a problem that exists at the intersection between trust and scale and that I believe that local nodes are the only real solution that can handle both.

Instead of trying to detect and block it, just disincentivize it.

Most AI spam on social media tries to exploit various systems intended to predict “good” content on the basis of a user’s past activity, by tracking reputation/karma/etc. Bots build up karma by posting a massive amount of innocuous (but usually insipid) content, then leverage that karma to increase the visibility of malicious content. Both halves of this process result in worse content than if the karma system didn’t exist in the first place.

Maybe it was silently assumed but nobody so far mentioned the endless stream of scrapers that go through my probably juicy but private instance. I‘m banning a new bot every week and by now they have switched to distributed actions. I get over 400 requests per hour by a couple ips for the same stuff with changing useragents because I wrote automated detection mechanisms. I might just make my instance login only.

"The fediverse" really can't. That's just the reality of a decentralized system. It's going to be up to individual instances to sort it out.

But that's a good thing, because what it means is that different instances can and will try different approaches, and between them, they'll sooner or later hit on the one(s) that will be most effective.

I think that being human scale is largely the appeal of the Fediverse. Each instance isn't meant to grow to the size of a centralized platform, but to be a relatively small community of people with some shared interests. I look at it similarly to the way IRC channels worked back in the day. You tend to have a group of people whom you interact with frequently and that's how you know they're human. If some bot enters the community then it becomes obvious very quickly.

The fediverse architecture was built from the beginning to allow instance-by-instance exercise of discretion to mute any systemic effects that could take over the network as a whole.

This was I think oriented toward limiting swarming behavior from trolls, but I think it also applies to AI bots.

Right now it seems that the Fediverses main protection is that it just isn’t a juicy enough target for wide scale spam and bad faith agenda pushers.

If you ask me they are already here right now, but I think it's not the architecture of the fediverse, but the judgment of individual mods that have let us down in this case.

Heavy moderation. Some real people will be flagged as bots, but it's the only way. A well-prepared prompt can be indistinguishable from human language. Registration questions will only block bots because they stop bot makers from registering thousands of accounts at once.

Servers with lax signup requirements will need to be blocked (temporarily?) when spammers find their way in. Detecting links and other promoted content can easily get rid of the commercial spammers, but it gets harder when you're trying to block political bot farms. Even still, Fediverse software lacks any kind of standard "firewall" against spam, as a few Japanese teenagers demonstrated with a script not that long ago, and the trolls spamming child porn before them.

On the wider Fediverse, it's quite possible to set up a Lemmy server consisting entirely of bots, or a whole network of them, first only interacting with each other but soon becoming part of the wider Fediverse.

Luckily, almost nobody cares about the Fediverse and the few bad actors I've seen don't care enough to actually put in any effort. If the Fediverse does ever grow to become a sizable influence on the internet, we're pretty much fucked.

I don't think there is any way to have a genuine "open forum" amongst complete strangers. There have always been human troll farms pushing narratives using sock puppet accounts, AI is just enabling it to reach new scales.

I actually am for echo chambers when it comes to social media, but one in which you only follow people you know or trust and ignore complete strangers and to make sure you get news and critical information from OUTSIDE social media, again with institutions you trust.

I have had similar thoughts, I think the answer ultimately lies in active mods that can really get to know a community and it's users and identify when users are pushing a narrative even if they can't confirm if they are a bot or not.

Also as @dessalines@lemmy.ml pointed out, user registrations. On startrek.website we have a question that is easy for a star trek fan to answer but not easy for a bot (although getting back to your concern, chatGPT probably would have no problem)

Same way email handles spam ...

What can be done? Smarter people can probably list plenty of things. But in the end, it's a constant race trying to out compete. And with LLMs/AI, you can literally train it on the system you want it to overcome with that express purpose and let it work out the "how" and you're back to square one again.

I think it can best be put in song

Or put another way: how do you make a bear proof trashcan that can defeat a bear but not the dumbest of humans?

As you said, a 44k monthly active users plateform is probably not worth investing time from spammers and agenda pushers.

If at some point we'll make it, we'll see. Seems like we are still quite far.