How do we deal with illegal content?

As usual, it's the same as email. There will need to be various sorts of spam filtering developed in order to keep the platform usable. In the meantime — if you see it, report it and delete it.

Suppose you open up your email and you see that you've received a piece of spam that contains CSAM (CP). You have not committed a crime — but you also mustn't keep it. So you report the spam to your email provider, and you delete it from your mailbox. If you're very diligent maybe you report it to NCMEC.

Suppose you run an email server. You're aware of the existence of spam (alas!) and you do your best to block spam using various technologies ranging from DNSBLs to ML classifiers. If someone on the Internet sends spam containing CSAM to a user on your server, you didn't send it; they did. The sender committed a crime. Your spam filters just didn't catch that particular instance. So when your user reports it to you, you improve your spam filters. And you delete it.

Suppose you run an email server. Your spam filters might include a reputation score for other email servers. When your filters notice that a large fraction of the messages from a particular server are spam, they switch to automatically block all mail from that server. Then even if that server tries to send spam to your users, the offending messages never even hit your server's disk.

Expect that as this platform matures, it will need many of the same sorts of spam-mitigation technology that email and other federated services have used.

I'm repeating "and you delete it" once again because that's important. You mustn't retain copies of illegal files even as training data for your spam classifiers. The big email providers & social media companies go to a bunch of effort to keep data about CSAM files, without having to keep the actual files.

If it's still on the internet you report them to law enforcement. But I'd bet that those intent on hosting those kinds of materials would have already started their own instances on the dark web and there sadly isn't much we can do in that case. Only way to deal with them seems to be law enforcement's approach of trapping those predators by posing as clients but then again, that's their job, not yours. What you can do is 1- defederate, 2- warn other instance admins and 3- report to the police.

Legality also totally subjective, it's a big gray area for sure

As an example, what if someone creates a Lemmy instance and starts creating a community around CP? Or human trafficking? How do we deal with it?

Can you elaborate on who "we" is?

Fortunately images and thumbnails uploaded by remote users are hosted on the remote instance, not yours.

You should focus on making sure that your instance's communities stay within the legality of your country, as well as flag and deal with any illegal behavior by your users.

For anything remote you have two options:

1. Block the whole instance (not recommended unless it's clear that the whole instance is dedicated to something that's illegal in your country or if they host something incredibly disturbing)
2. Click on "Remove" on the main page of a remote community. This will remove the remote community and make it inaccessible to local users but keep you federated to their instance.

You cannot control what other servers do. There will be servers out there hosting illegal stuff. But that's not something you or I need to fix, that's where law enforcement needs to be involved. The only thing you can do is block. If it's something serious like CP or human traficking, grab any logs you might have from them, report to authorities, purge content from database and block instance.