Welcome to Incremental Social! Learn more about this project here!
Check out lemmyverse to find more communities to join from here!

‘We definitely messed up’: why did Google AI tool make offensive historical images?

Brin’s “We definitely messed up.”, at an AI “hackathon” event on 2 March, followed a slew of social media posts showing Gemini’s image generation tool depicting a variety of historical figures – including popes, founding fathers of the US and, most excruciatingly, German second world war soldiers – as people of colour.

alicehughes ,

Maybe there is a ban from ChatGPT

Kolanaki ,
@Kolanaki@yiffit.net avatar

What? Gemini has an image gen tool? That fucker told me it didn't when I asked! Dumbass AI don't even know what it can do... SMH

watersnipje ,

They switched off image generation after these issues, so it (correctly) said that it couldn’t generate images at the time.

cupcakezealot ,
@cupcakezealot@lemmy.blahaj.zone avatar

was it really offensive or was it just "target selling pride clothes during pride month" offensive?

helenslunch ,
@helenslunch@feddit.nl avatar

I don't know that "offensive" is the right word. More just "shitty" and "lazy".

Like they took the time out to teach it "diversity" but couldn't bother to train it past "diversity = people who are not white" or to acknowledge when the user is asking specifically for a white person or a different region or time period.

Faydaikin ,
@Faydaikin@beehaw.org avatar

I, for one, welcome Japanese George Washington, Indian Hitler and Inuit Ghandi to our historical database.

Malgas ,

Jojo Rabbit featured Jewish Maori Hitler and was very well received.

CanadaPlus ,

I think the lesson here is that political correctness isn't very machine learnable. Human history and modern social concerns are very complex in a precise way and really should be addressed with conventional rules and algorithms. Or manually, but that's obviously not scalable at all.

GadgeteerZA ,
@GadgeteerZA@beehaw.org avatar

It's not just historical. I'm a white male and I prompted Gemini to create images for me if a middle aged white man building a Lego set etc. Only one image was a white male and two of the others wrecan Indian and a Black male. Why when I asked for a white male. It was an image I wanted to share to my family. Why would Gemini go off the prompt? I did not ask for diversity, nor was it expected for that purpose, and I got no other options for images which I could consider so it was a fail.

TheAlbatross ,

Could you elaborate on the use case you're describing? You were trying to make an image of a middle aged white man building Lego for your family?

GadgeteerZA ,
@GadgeteerZA@beehaw.org avatar

Yes, but it does not really matter what the rest of the prompt detail was? The point was, it was supposed to me an image of me doing an activity. I'd clearly prompted for a white man, but it gave me two other images that were completely not that. Why was Gemini deviating from specific prompts like that? Seems the identical issue to the case with the Nazis, just introducing variations completely of its own.

TheAlbatross ,

Yeah yeah sure sure but why were you generating an image of a middle aged white man building Lego for your family? I'm baffled.

GadgeteerZA ,
@GadgeteerZA@beehaw.org avatar

That is really just not relevant at all to the discussion here, but to satisfy your curiosity, I'm busy building a Lego model that a family member sent me, so the generated AI photo was supposed to depict someone that looked vaguely like me building such a Lego model. I used Bing in the past, and it has usually delivered 4 usable choices. Fact that Google gave me something that was distinctly NOT what I asked for, means it is messing with the specifics that are asked for.

TheAlbatross ,

Why use an AI? Just like... take a selfie

letsgo ,

So, what you're saying is that white people shouldn't use AI?

TheAlbatross ,

It would appear that is exactly what I'm saying as long as the reader lacked any reading comprehension skills.

memfree ,

I'm not the lego person, but I am not taking that selfie because: 1) I don't want to clean the house to make it look all nice before judgey relatives critique the pic, 2) my phone is old and all its pics are kinda fish-eyed, 3) I don't actually want to spend the time doing the task right now when AI can get me an image in seconds.

Ephera ,

The problem is that the training data is biased and these AIs pick up on biases extremely well and reinforce them.

For example, people of color tend to post fewer pictures of themselves on the internet, mostly because remaining anonymous is preferable to experiencing racism.
So, if you've then got a journalistic picture, like from the food banks mentioned in the article, suddenly there will be relatively many people of color there, compared to what the AI has seen from its other training data.
As a result, it will store that one of the defining features of how a food bank looks like, is that it has people of color there.

To try to combat these biases, the bandaid fix is to prefix your query with instructions to generate diverse pictures. As in, literally prefix. They're simply putting words in your mouth (which is industry standard).

GadgeteerZA ,
@GadgeteerZA@beehaw.org avatar

Sometimes you do want something specific. I can understand if someone just asked for a person x, y, z and then gets a broader selection of men, women, young, old, black or white. But if one asks for a middle-aged white man, I would not expect it to respond with a young, Black women, just to have variety. I'd expect other non-stated variables to be varied. It's like asking for a scene of specifically leafy green trees, then I would not expect to see a whole lot of leafless trees.

Ephera ,

Yeah, the problem with that is that there's no logic behind it. To the AI, "white person" is equally as white as "banker". It only knows what a white person looks like, because it's been shown lots of pictures of white people and those were labeled "white person". Similarly, it's been shown lots of pictures of white people and those were labeled "banker".

There is a way to fix that, which is to introduce a logic before the query is sent to the AI. It needs to be detected whether your query contains explicit reference to skin color (or similar), and if so, that query prefix needs to be left out.

Where it gets wild, is that you can ask the AI whether your query contains such explicit references to skin color and it will genuinely do quite well at answering that correctly, because text processing is its core competence.
But then it will answer you "Yes." or "No." or "Potato chips." and you have to program the condition to then leave out the query prefix.

GadgeteerZA ,
@GadgeteerZA@beehaw.org avatar

Yes, it could be that, and may explain why the Nazi images came out like they did. But it sounded more like to me, Google was forcing diversity into the images deliberately. But sometimes that does not make sense. For general requests, yes. Otherwise they can just as well decide that grass should not always be green or brown, but sometimes also just make it blue or purple for variety.

scrubbles ,
@scrubbles@poptalk.scrubbles.tech avatar

Nah, in this case I think it's a classic case of over correction and prompt manipulation. The bus you're talking about is right, so to try to combat that they and other ai companies manipulate your prompt before feeding it to the llm. I'm very sure they are stripping out white male and or subbing in different ethnicities to try to cover the bias

GluWu ,

TFW you accidentally leave the hidden diversity LoRa weight at 1.00.

frogmint ,

For example, people of color tend to post fewer pictures of themselves on the internet, mostly because remaining anonymous is preferable to experiencing racism.

That is quite the bold statement. Source?

Ephera ,

I don't think I came up with that myself, but yeah, I've got nothing. Would have been multiple years, since I've read about that.
Maybe strike the "mostly", but then it seemed logical enough to me that this would be a factor, similar to how some women will avoid revealing their gender (in certain contexts on the internet) to steer clear from sexual harassment.
For that last part, I can refer you to a woman from which I've heard first-hand that she avoids voice chat in games, because of that.

NoLifeKing ,
@NoLifeKing@ani.social avatar

Why is that a problem? These things happened and why shouldn't the "ai" get images of it?

Daxtron2 ,

The issue is not that it can generate the images, it's that the filtering a pre prompt for Gemini was coercing the images to include forced diversity into the gens. So asking for 1940s German soldier would give you multiracial Nazis, even though that obviously doesn't make sense and it's explicitly not what was asked for.

NoLifeKing ,
@NoLifeKing@ani.social avatar

Lmao

Daxtron2 ,

It is a pretty silly scenario lol, I personally don't really care but I can understand why they implemented the safeguard but also why it's overly aggressive and needs to be tuned more.

NoLifeKing ,
@NoLifeKing@ani.social avatar

Why even put a save guard in place? Nobody needs it anyway.

entropicdrift ,
@entropicdrift@lemmy.sdf.org avatar

Corporations making AI tools available to the general public are under a ton of scrutiny right now and are kinda in a "damned if you do, damned if you don't" situation. At the other extreme, if they completely uncensored it, the big controversial story would be that pedophiles are generating images of child porn or some other equally heinous shit.

These are the inevitable growing pains of a new industry with a ton of hype and PR behind it.

maynarkh ,

TBH it's just a byproduct of the "everything is a service, nothing is a product" age of the industry. Google is responsible for what random people do with their products.

Kichae ,

If you create an image generator that always returns clean cut white men whenever you ask it to produce a "doctor" or a "business man", but only ever spits out black when when you ask for a picture of someone cleaning, your PR department is going to have a bad time.

t3rmit3 ,

And even worse, it actually reinforces that image within users.

autotldr Bot ,

🤖 I'm a bot that provides automatic summaries for articles:

Click here to see the summary

Brin’s comments, at an AI “hackathon” event on 2 March, follow a slew of social media posts showing Gemini’s image generation tool depicting a variety of historical figures – including popes, founding fathers of the US and, most excruciatingly, German second world war soldiers – as people of colour.

The pictures, as well as Gemini chatbot responses that vacillated over whether libertarians or Stalin had caused the greater harm, led to an explosion of negative commentary from figures such as Elon Musk who saw it as another front in the culture wars.

But it follows a similar pattern to an uncovered system prompt for OpenAI’s Dall-E, which was instructed to “diversify depictions of ALL images with people to include DESCENT and GENDER for EACH person using direct term”.

Dame Wendy Hall, a professor of computer science at the University of Southampton and a member of the UN’s advisory body on AI, says Google was under pressure to respond to OpenAI’s runaway success with ChatGPT and Dall-E and simply did not test the technology thoroughly enough.

Hall says Gemini’s failings will at least help focus the AI safety debate on immediate concerns such as combating deepfakes rather than the existential threats that have been a prominent feature of discussion around the technology’s potential pitfalls.

Dan Ives, an analyst at the US financial services firm Wedbush Securities, says Pichai’s job may not be under immediate threat but investors want to see multibillion-dollar AI investments succeed.

Saved 78% of original text.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • technology@beehaw.org
  • random
  • incremental_games
  • meta
  • All magazines
    •