1y ago

AI-generated child sexual abuse images could flood the internet. A watchdog is calling for action

apnews.com AI-generated child sexual abuse images could flood the internet. A watchdog is calling for action

A new report warns that the proliferation of child sexual abuse images on the internet could become much worse if something is not done to put controls on artificial intelligence tools that generate deepfake photos.

Technology @lemmy.world

apnews.com /article/ai-artificial-intelligence-child-sexual-abuse-c8f17de56d41f05f55286eb6177138d2

81 27

You're viewing a single thread.

114 comments

vastly expands the pool of potential victims
I'm not brave enough at the moment to say it isn't some kind of crime, but creating such images (as opposed to spamming them everywhere, using them for blackmail, or whatever) doesn't seem to be a crime that involves any victims.
- My bigger concern is the normalization of and exposure to those ideas and concepts (sexualization of children). That’s also why I dislike loli/shota media, despite it being fictional.
  That said, I still think it’s a much better alternative to CSAM and especially to actually harming a child for those who have those desires due to trauma or mental illness. Though I’m not sure if easy, open access is entirely safe, either.
  
  My bigger concern is the normalization of and exposure to those ideas and concepts
  The same concern has been behind attempts to restrict/ban violent video games, and films before that, and books before that. Despite generations of trying, I don't think a causal link has ever been established.
  
  On the flip side, studies haven’t come to a single consensus of viewing cp leading to reduced violence by individuals either.
  While a full-ban infringes upon individual rights of expression and speech, and may impede in previous victims viewing it as an alternative, I’m not sure if a laissez faire approach is the best option, either.
  Especially for material that A) depicts abuse and B) is harder to distinguish between fiction and reality (AI generated content), the risk of psychological harm to individuals without existing trauma or fetishes is very real. I stand by this fact for violent/unethical media as well.
  
  In that regard, with books, games, movies, and drawings it's easy to discern fantasy from reality. With an AI generated photograph that becomes increasingly difficult to do.
  
  with books, games, movies, and drawings it’s easy to discern fantasy from reality
  I don't think it is easy with movies or books, unless you are certain of the source.
  Either way, we don't have a causal link.
  
  I don't think it's the same concern. It's not that people will become pedophiles or act on it more because of the normalization and exposure. It's people will see less of a problem with the sexualization of children. The parallel being the amount of violence we are OK being depicted. The difference being we can only emulate in a personal level the sexual side.
  Maybe there's the argument that violence is escapist, sexual desire is ever present and porn is addictive.
  
  A teenager who plays a violent video game is not engaging in an act of violence as recognised by his brain. He is not going into a fight or flight response and getting trauma from the experience as he would in a real fight. His brain doesn't think he's in a fight.
  When you masturbate, your body goes through the same chemical and neurological processes as if you were really having sex.
  
  That's not entirely true. Some studies have shown that stuff we watch influences our decisions and behaviour.
  This article gives an overview over some of the more accepted research done in the area:
  Pornography Use and Psychological Science: A Call for Consideration
  
  For "normalisation of sexualisation of children" go ask the people organizing child beauty pageants.
  
  So you agree? It shouldn't be produced because it can be used to normalise the sexualisation of children or even groom them.
  
  I didn't say that I agree, I just pointed out that there are way more prominent ways this sexualisation is done.
  I also don't agree with the headline of the article that this kind of pictures will somehow "flood" the internet. It might flood their hidden nieches for being cheap and plentiful, but I don't think they will pop up increasingly in any normal users everyday browsing activities.
- I'm brave enough to say what I am sure some people are thinking.
  If a pedophile can have access to a machine that generates endless child porn for them, completely cutting off the market for the "real thing", then maybe that's a step in a positive direction. Very far from perfect but better than the status quo.
  The ideal ultimate solution is to develop a treatment that pedophiles can use to just stop being pedophiles entirely. I bet most pedophiles would jump on such a thing. But until that magical day maybe let's explore options that reduce the harm done to actually real children in the immediate term.
  
  Some psychologists agree with you. Others say it would only make the problem worse, making them want to escalate. Definitely one that I'm letting the professionals debate on and I'll go with their opinion
  
  The consensus is that access to porn lowers rape. Does this extend to pedophilia? Probably, but not proven, in large part because of obvious reasons.. testing this theory is super duper internationally illegal
  
  Some can't be cured by their own admission, its seems for some it is a deep seated sexual deviation that was probably established in very eaely life.
  
  "By their own admission" doesn't necessarily hold any weight, though. They're not experts, and even if they were experts they can still be wrong. We've got treatments for a variety of psychological disorders these days, with varying efficacy, I don't think it's unreasonable to expect that someday we'll make progress on this sort of thing too.
  
  If we have the tech to change people's brains in such a way, what else will it gets used on? Ethically, there's differences between pedophilia and other sexual preferences but I don't know if there's any biological differences. Given conversion camps are already a problem despite not working, I can't see how they wouldn't become a bigger problem if they did actually do what they claimed...
  
  We have the tech right now to shoot people dead. It's just tech, create laws about what's acceptable to do with it and what's not.
  
  Even when conversion camp stuff is illegal, people still try it. Even if just DIY abuse (which is also illegal and still common). Simply not having the tech may be preferable to having the tech/knowledge, making it illegal to use it, and then people use it illegally anyways. If child abuse in general was something that could have just not been invented, not inventing it would be far superior to inventing it and then making it illegal.
  
  oh yeah cause that's working out oh so perfectly
  
  This thread is about how to regulate AI-generated child porn, that's going to require creating laws too.
  What's your proposed alternative to using laws to regulate tech?
  
  I was just saying that this tech that shoots people dead isn't really regulated all too well in certain parts of the world.
  
  Well, mainly just one. There are lots of other places in the world that do okay at it.
  
  You have offenders saying if you let me out I will do it again you don't think they are experts on their own feelings or behaviour), just like pscyhopaths some can't be fixed. As the word suggests pedophile means loving kids. While some are just nasty ass people, many see it as their sexual orientation (still nasty to harm kids obviously). Like being straighr or being gay. it can be hard wired. i'm sure with early intervention before 7 you might fix deviation or at least teach restraint from acting in it... but they have formed a concrete ldentity. I think larger strides will be made by protecting kids from sexual abuse at an early age than therapy after the fact trying to fix brain wiring
- but creating such images (as opposed to spamming them everywhere, using them for blackmail, or whatever) doesn’t seem to be a crime that involves any victims.
  Well, there's all the children whose photos were used for the training data. I'd consider them victims, since AIs can't produce truly new images, so real human victims were needed in order to make AI images possible. And it's been established that AIs need to be trained on new human-made content in order to develop, as the images become distorted when trained on AI-generated content, so unless the paedophiles can be convinced to be satisfied with the AIs as they currently are instead of wanting better/more varied child abuse images next year, a whole lot more real children will need to be abused and photographed in order to improve the AI.
  
  There's a lot of misconceptions about AI image generators in here.
  They can indeed generate "truly new images", ask an image generator for an image of something that definitely doesn't exist in its training set and it'll likely be able to come up with something like that for you. Most importantly for purposes of this discussion, you don't actually need to have any images specifically of child abuse in a model's training set in order to train it well enough to produce images of child abuse. Train a model with a bunch of regular porn and a bunch of ordinary images of children and I expect it'll figure out how to make images of children in sexual situations if you ask it to.
  This has been known for years. These AIs are capable of "understanding" the things they're trained on and creating novel interpretations of those things.
  There was an article recently that showed if you trained many generations of AIs on just the outputs of previous generations you got degraded performance over time, but that's a pretty specific scenario that doesn't match what's being done in real life. In real life synthetic training data (ie, AI-generated training data) can be very useful for expanding the capabilities of AI as long as it's well-curated (humans need to select good outputs and ensure they're described correctly) and ideally has some of the earlier training set's original data included as well.
  
  So basically, in order to have AIs that make better child porn, there needs to be humans willing to go through vast quantities of AI generated child porn in order to properly curate the content for the AI. Since this labour is likely to be farmed out to innocent people in developing countries, being paid slave wages, I think it would be fair to add them to the list of potential victims of the creation of child porn AIs.
  
  Since this labour is likely to be farmed out to innocent people in developing countries
  You don't quite seem to understand how easy it is to train these AI models, and because of that, you're missing a critical point - with open-source technologies like Stable Diffusion, which has models that can be refined and run on a consumer-grade graphics card, the people using models to generate images and the people creating and refining those models are the same people. People who want to generate brand new pokemon sprites can train a model on all the pokemon sprites until it looks good. A few absolute galaxy-brain nerds who want to generate MIDI spectrograms from a text description and convert the output into audio... can apparently do that. And of course, people who want to generate lots of hentai or photorealistic porn can create and fine-tune a model, or multiple models, all by themselves (I won't link any of these, but hundreds are readily available, and thousands exist in total)
  In other words, people who already consume CSAM are the people working on models for generating CP, and a subset of those have definitely been trying to make it work with only legal images so that the model itself can be distributed and used without breaking any laws, maybe even hiding in plain sight pretending it's not for making CP. Someone else out there with a different set of fucked-up desires has probably trained a model on gore and snuff images and then used it to create "photos" of people they hate as mutilated messes. There's sick people of all kinds all over the place, and the jury's unfortunately still out on whether this new tool actually causes harm when used in such a manner, or if it's just the newest way they can express their deviance. We don't know yet.
  But this genie is already out of the bottle. Banning the use of this technology for specific, narrow use cases just isn't going to be effective without banning AI image generation entirely, and we're past the point where that's feasible. Image generation is a powerful tool that's not going away; it's on us now to figure out what we really believe about harm, health, and personal freedom, and what we want a society with this tool to look like.
  Personally, I'm of a mind that if all the data going into the model is legally obtained, anything generated should be considered artistic expression. A person had a thought, then put their thoughts into a tool, which made a picture of those thoughts. No matter how repulsive those thoughts were, I think throwing people in prison for that kind of expression is thought-crime. There's public obscenity at play, of course, but only once they take the step of showing it to other people. If it's just for themselves, and nobody else sees it, who is harmed? Even if it does turn out that it harms the person generating the images (which wouldn't surprise me), that makes it a health issue, like drugs or other addictions, not something to criminalize.
  
  Except if we're talking about allowing regulated use of AI-generated child porn as a treatment for paedophilia (which is the core discussion in this thread, whether AI-generated child porn could be helpful), then it cannot be left to paedophiles themselves to create their own models based on the honour system of promising that nobody will use photos of real children in the training data. Just like absolutely no aspect of AI can be left unregulated, trusting on everyone to behave honourably... because so far, nobody has been. Not just in the field of AI-generated child porn. All of AI has been developed with an astounding level of unethical behaviour, and it's nothing short of complete naivety to believe that anyone is going to behave ethically going forward. If AI is to be considered in any way ethical, then it needs regulation beyond simply asking people to pinky swear that they will only use legally and ethically obtained content for training. Regulation requires oversight and enforcement, otherwise regulation is meaningless. And you can't regulate child porn AI without having innocent human beings subjected to the inputs and outputs to ensure regulations are being followed.
  
  As I said above, though, you don't need to make a model that's specifically "for child porn" in order for it to be able to generate child porn. There are already probably plenty of models that know what children look like and also know what porn looks like, made simply by teaching a model about lots of diverse subjects that happened to include both of those subject areas in them. You can even make new models by merging two existing models together or by adding more training to an existing model, so you wouldn't even need to have those images be part of the same training run.
  I obviously haven't ever tried generating child porn, but I fired up my local Stable Diffusion with the Cyberrealistic model and generated a toddler on the moon and a toddler riding a lion. I'm reasonably confident that the model wasn't literally trained with images of toddlers in space suits or toddlers riding large wild predators, it was trained on those concepts separately and was able to figure out for itself how to combine them. Notice how it was able to figure out that a toddler on the moon would be in a space suit and re-proportioned the space suit accordingly, and that a saddle used by a toddler would probably have handlebars (I'm guessing it has a bunch of images of toddlers riding ponies that it got that idea from).
  
  @the_third@feddit.de did a very good post above explaining the problem with the idea of thinking AI-generated child porn would be as simple as asking for non-abusive photos of children to be combined with photos of adult porn. The AI needs to know what each component of the image should look like. AI knows what toddlers look like, and it knows what a lion looks like. Where do you propose the photos of child genitals should come from in order to create these ethical AI-generated child abuse images?
  
  Medical textbooks.
  
  Probably fine for child abuse porn in a drawn style, like loli, but probably not sufficient for the photorealistic porn that it is supposedly intended to replace.
  This is ultimately the flaw in your argument that an AI can produce "new" works because it doesn't need to have seen a toddler riding a lion on the moon to be able to produce that image. If you didn't give AI photos of lions, it would never be able to create a lion. If you never gave it photos of toddlers, it would never be able to do a picture of a toddler. And if there were no photos of the moon in its training data, it would be incapable of producing the moon. It cannot create things it has not seen. It can only arrange things it has seen in combinations that may or may not have been previously thought of (with billions of images in the training data, you can't say there isn't a Photoshopped photo of a toddler on the moon in there.)
  Without actual child porn in its training data, it would never be able to produce any, because even when it's capable of piecing individual elements together into a "new" piece (basically an advanced collage), if there's no images of children being abused in the training data, it's not going to be able to piece it together by putting a child's head on an adult's naked body, and result in anything that's more satisfactory for paedophiles than actual photos of children. There is therefore no ethical means of producing AI-generated child porn, and therefore it is not an ethical alternative. Somewhere in the chain, there are still children being abused.
  
  I think you haven't made much use of image-generating AIs. They're quite capable of reinterpreting images into different styles. A very common use case for me is to draw a sketch of something and then tell an image AI to turn it photorealistic. The "automated collage" approach you describe is simply not how they work, it's a common misconception. Image AIs very much can create imagery of things that weren't explicitly in their training set, they're not just regurgitating pasted-together snippets.
  You're also assuming that there are no literal photographs of children's genitals in medical literature. Again, I haven't exactly gone looking, but I'm sure there are some out there. Doctors can't afford to be prudish.
  And finally, you can get plenty pornographic without even specifically showing off genitals.
  
  Dude how are you going to regulate AI?!!
  Like by all means go after people harming others, but man... You realize you have no control right? Black markets THRIVE in this world. Like fuck, they can't even keep drugs out of prison.
  
  The fact that regulation is imperfect doesn't mean the solution is to do absolutely nothing and let AI be used by bad actors (including those feeding real child porn into it) with impunity.
  The entire conversation in the comments here have been "well if AI child porn stops paedophiles from hurting real children, then it has a use", but in order to prevent real children being used for the training data, it has to be regulated. In fact, you'll find that the majority of Beehaw users (since I notice you are from a different instance) are in favour of AI being properly regulated. "Some people will break the law anyway" is no excuse not to have laws.
  
  Sorry, I really don't support your perspective. If you have reasonable suspicion to believe real child porn is being used in these models prove it to a judge and get a warrant. If someone is a bad actor to the point of criminal intent, prove it to a judge, and bring it to court.
  
  That's not how the law works.
  
  Wrong.
  
  They don't have a plan they just want to blissfully hand law enforcement more power to spy on citizens and toss non violent offenders in prison. War on drugs called, wants its script back.
- Another worry could be: how do you know if it’s a real victim who needs help, or an AI generated image.
  
  If we had access to the original model, we could give it the same seed and prompt and get the exact image back. Or, we could mandate techniques like statistical fingerprinting. Without the model though, it's proven to be mathematically impossible the better models get in the coming years - and what do you do if they take a real image, compress it into an embedding, then reassemble it?
- Many "AI generated" images are actually very close to individual images from their training data so it's debatable how much difference there is between looking at a generated image and just looking at an image from its training data in some cases at least.
  
  That's a symptom of overfitting, which requires the image to be repeated in the training hundreds or even thousands of times. That generally only happened in earlier image generation models, more "modern" ones ("modern" in this case being measured in months because this is such a fast-developing technology) have much better curation of their training sets to avoid exactly that sort of thing. Nobody wants AI image generators that replicate images from their training sets, what would be the point?
  So if you want to find an image model that gives you a close duplicate of an existing image of child abuse, you'll need to find one that was sloppily trained with a training set that included hundreds of duplicates of child abuse imagery. I kind of doubt you'll be able to find one of those.

114 comments