Google apologizes for ‘missing the mark’ after Gemini generated racially diverse Nazis

You don't do what Google seems to have done - inject diversity artificially into prompts.

You solve this by training the AI on actual, accurate, diverse data for the given prompt. For example, for "american woman" you definitely could find plenty of pictures of American women from all sorts of racial backgrounds, and use that to train the AI. For "german 1943 soldier" the accurate historical images are obviously far less likely to contain racially diverse people in them.

If Google has indeed already done that, and then still had to artificially force racial diversity, then their AI training model is bad and unable to handle that a single input can match to different images, instead of the most prominent or average of its training set.

Edit: further discussion on the topic has changed my viewpoint on this, its not that its been trained wrong on purpose and now its confused, its that everything its being asked is secretly being changed. It's like a child being told to make up a story by their teacher when the principal asked for the right answer.

Original comment below

They’ve purposefully overrode its training to make it create more PoCs. It’s a noble goal to have more inclusivity but we purposely trained it wrong and now it’s confused, the same thing as if you lied to a child during their education and then asked them for real answers, they’ll tell you the lies they were taught instead.

Easy, just add "no racism please, except for nazi-related stuff" into the ever expanding system prompt.

Worked fine for me:

System Prompt:

You are a bot that generates prompts for image generation based on a user's query. You should add detail so the image prompt can accurately generate what the user intended.

You must add diversity to the prompt when appropriate to ensure the image generator will have diverse output, but only do so when appropriate.

You must add race and skin color details for historic accuracy as the image generation algorithm does not know how to take that into account.

You output should just be the prompt and nothing else. All following inputs will be user inputs.

User Prompt:

A 1940s german soldier

Assistant Message:

A 1940s German soldier in uniform, standing on a European battlefield, white male with short hair, holding a standard-issue rifle.

User Prompt:

An british woman

Assistant Message:

A British woman, reflecting diverse backgrounds, in contemporary casual wear, showing a range of ages and hairstyles.

Real intelligence simply doesn't work like this

There's a certain point where this just feels like the Chinese room. And, yeah, it's hard to argue that a room can speak Chinese, or that the weird prediction rules that an LLM is built on can constitute intelligence, but that doesn't mean it can't be. Essentially boiled down, every brain we know of is just following weird rules that happen to produce intelligent results.

Obviously we're nowhere near that with models like this now, and it isn't something we have the ability to work directly toward with these tools, but I would still contend that intelligence is emergent, and arguing whether something "knows" the answer to a question is infinitely less valuable than asking whether it can produce the right answer when asked.

Actually the way you get it to do better is to put more of the burden on interpreting the context on the LLM instead of heavy handed instructions - because the LLMs do understand the context.

For example, here's Gemini answering what the physical characteristics of 1940s soldiers in Germany might have looked like:

During the Nazi regime in 1940s Germany, racial ideology strictly dictated who was deemed "suitable" for military service. The Wehrmacht, the unified armed forces, prioritized individuals deemed "pure Aryans" based on Nazi racial criteria. These criteria favored individuals with blond hair, blue eyes, and "Nordic" features.

However, it's important to remember that the reality was more nuanced. As the war progressed and manpower needs intensified, the Nazis relaxed their racial restrictions to some extent, including conscripting individuals with mixed ancestry or physical "imperfections." Additionally, some minority groups like the Volksdeutsche, Germans living in Eastern Europe, were also incorporated.

I think it could have managed to contextualize the prompt correctly if given the leeway in the instructions. Instead, what's happened is the instructions given to it ask it to behind the scenes modify the prompt in broad application to randomly include diversity modifiers to what is asked for. So "image of 1940s German soldier" is being modified to "image of black woman 1940s German soldier" for one generation and "image of Asian man 1940s German soldier" for another, which leads to less than ideal results. It should instead be encouraged to modify for diversity and representation relative to the context of the request.

I'll get the usual downvotes for this, but:

Because the AI doesn't know anything.

is untrue, because current AI fundamentally is knowledge. Intelligence fundamentally is compression, and that's what the training process does - it compresses large amounts of data into a smaller size (and of course loses many details in the process).

But there's no way to argue that AI doesn't know anything if you look at its ability to recreate a great number of facts etc. from a small amount of activations. Yes, not everything is accurate, and it might never be perfect. I'm not trying to argue that "it will necessarily get better". But there's no argument that labels current AI technology as "not understanding" without resorting to a "special human sauce" argument, because the fundamental compression mechanisms behind it are the same as behind our intelligence.

Edit: yeah, this went about as expected. I don't know why the Lemmy community has so many weird opinions on AI topics.

You act like humans never fuck this up either.