Welcome to Incremental Social! Learn more about this project here!
Check out lemmyverse to find more communities to join from here!

menemen ,
@menemen@lemmy.world avatar

Ah, so it only affects searching from the US for now. I already wondered why I couldn't reproduce the stuff I saw here and why I didn't really see a change at my Google results.

billiam0202 ,

You probably live in one of those "socialist" countries that has, like, consumer protection laws and stuff.

adam_y ,
@adam_y@lemmy.world avatar

Can we swap out the word "hallucinations" for the word "bullshit"?

I think all AI/LLM stuf should be prefaced as "someone down the pub said..."

So, "someone down the pub said you can eat rocks" or, "someone down the pub said you should put glue on your pizza".

Hallucinations are cool, shit like this is worthless.

kbin_space_program ,

Google search isnt a hallucination now though.

It instead proves that LLMs just reproduce from the model they are supplied with. For example, the "glue on pizza" comment is from a reddit user called FuckSmith roughly 11 years ago.

billiam0202 ,

Without knowing what specific comment it was, I'm going to guess it was on how advertisers make pizza look better in ads than real life?

kbin_space_program ,

Here is one news source reporting on the fun:

https://www.dailydot.com/debug/google-search-results-reddit-pizza-glue-cheese/

It was an actual shitpost. I had originally assumed the same as you, given that there are a few bloggers and youtubers who go through the tricks used for food photography.

AhismaMiasma ,

No, it was entirely a shitpost.

Original was deleted this week because of the news.

https://old.reddit.com/r/singularity/comments/1cyu29p/well_done_ufucksmith/

otacon239 ,

Nope. They were just trolling and rocking around. It was obvious sarcasm:

https://www.reddit.com/r/Pizza/comments/1a19s0/comment/c8t7bbp/

DarkThoughts ,

It instead proves that LLMs just reproduce from the model they are supplied with.

What do you mean by that? This isn't some secret but literally how LLMs work. lol
What people mean by hallucinating is when LLMs "create" facts that aren't any. Be it this genius recipe of glue pizza, or any other wild combination of its model's source material. The whole cooking thing is a great analogy actually because it's like all of their fed information are the ingredients, and it just spits out various recipes based on those ingredients, without any guarantee that it is actually edible.

kbin_space_program ,

There are a lot of people, including google itself, claiming that this behaviour is an isolated and basically blamed users for trolling them.

https://www.bbc.com/news/articles/cd11gzejgz4o

I was working on the concept of "hallucinations" being things returned that are unrelated to the input query, not directly part of the model as with the glue-pizza.

DarkThoughts ,

Your link does not match your statement.

kbin_space_program ,

A Google spokesperson told the BBC they were "isolated examples".

Some of the answers appeared to be based on Reddit comments or articles written by satirical site, The Onion.

But Google insisted the feature was generally working well.

"The examples we've seen are generally very uncommon queries, and aren’t representative of most people’s experiences," it said in a statement.

It said it had taken action where "policy violations" were identified and was using them to refine its systems.

That's precisely what they are saying.

DarkThoughts ,

I'm sorry but reading this as "Google blames users for trolling them" is either pure mental gymnastics or mental illness.

richieadler ,

This isn't some secret but literally how LLMs work. lol

Yeah, but John Q. Public reads AI and thinks HAL 9000 and Skynet, and no additional will convince them otherwise.

Eheran ,

No, hallucination is a really good term. It can be super confident and seemingly correct but still completely made up.

kbin_space_program ,

It is, but it isnt applicable in at least the glue-pizza situation as the probable source comment has been found on reddit.

A better use of the term might be how when you try to get Bing's image creator to make "Battletech" art, you just mostly get really obvious Warhammer 40k Space Marines and occasionally Iron Maiden album art.

barsoap ,

That's not hallucinations (in particular), that's concept bleed. Try the following:

  1. Acquire a human experimental subject. Ask them:
  2. What colour is snow?
  3. What colour is the fridge (point to a white fridge)?
  4. What do cows drink?

...and hear them answer "milk". "White, cold, drink, cow" are all wired to "milk" in our heads logic comes later. It's quite a bit harder to trick humans with this than AIs because we do have the capacity to double-check but if you simply want to bend an answer, not have it be completely nonsensical, it's quite easy.

Also your 40k or Iron Maiden result might very well still be Battletech. E.g. when it comes to image composition. Another explanation would be low resolution in the prompt encoding, that'd be similar to boomers calling your PS5 a Nintendo. Most likely though it has only seen two or three Battletech images (face it, it's not that popular in comparison) and thought "eh looks like a Nintendo that's where I'll store it", Humans and current-gen AI are different in principle in that regard as we can come up with encoding strategies, they can't. Something something T3 systems and need for exponential amounts of data.

snooggums ,
@snooggums@midwest.social avatar

That is just being WRONG.

yukijoou ,

for it to "hallucinate" things, it would have to believe in what it's saying. ai is unable to think - so it cannot hallucinate

Eheran ,

So how do you prove it can't think? Or that you actually can?

yukijoou ,

because it's a text generation machine..? i mean, i wouldn't say i can prove it, but i don't think anyone can prove it's capable of thinking, much less of reasoning

like, it can string together a coherent sentence thanks to well crafted equations, sure, but i wouldn't qualify that as "thinking", though i guess the definition of "thinking" is debatable

Eheran ,

It can tell you how to stack things on top of each other the best way to get a high tower. Etc.

Those are not random sentences. If you can not define thinking in a way this machine fails at, then stop saying it does not think.

Aceticon ,

A parrot can be trained to tell you how to stack things on top of each other the best way to get a high tower.

This is just an electronic parrot, millions of times faster to train than the biological parrot, specialized in repetition alone (can't really do anything else a parrot can) and which has been trained on billions of texts.

You're confusing one specific form in which humans externally express cogniscence with the actual cogniscence itself: just because intelligence can produce some forms of textual communication doesn't mean that the relationship holds in the opposite direction and such forms of textual communication require intelligence, or if you will, just because you can photograph a real pizza to get a picture of a pizza doesn't mean a picture of a pizza is actually of a real pizza and not something with glue to make it look like it has stringy melted cheese.

Eheran ,

Again, it is absolutely capable to come up with it's own logical stuff, hence my example. Stop saying it just copies existing stuff, that is simply wrong.

Aceticon ,

Sure, whatever.

Eheran ,

Amazing reply, given the context.

Aceticon ,

I'm actually a domain expert on AI whilst your "assertive denial without a single counter-argument" answer to my simplified explanation together with your "understanding" of the subject matter shown in the post before that one, shows you're at the peak of the Dunning-Krugger curve on this domain and also that you do not use analytical thinking or the scientific method in any way form or shape when analysing a subject.

There is literally no point in explaining anything to somebody who reasons like that and is at that point of that curve.

You keep your strongly held "common sense" beliefs and I'll keep from wasting any more of my time.

Eheran ,

this paper clearly says it is capable of original thought. It also "speaks" of it in high regard in other things.

Aceticon ,

Re-read it: it says AI is capable of "originality" and does not mention "thought" at all.

You're the one presuming that "originality" requires cognition and hence understood "originality" as meaning "original thought" even though they're different concepts (specifically the latter is a subset of the former).

In your interpretation of that paper you did the exact same logical mistake as you seem to be doing in your interpretation of LLMs - you made assumptions backed only by gut feeling thus taking a leap to reach a conclusion ultimately supported only by your gut feeling.

Eheran ,

"electronic parrot" and outperforming almost all humans in creativity and originality is an extreme contrast to me, regardless of my misuse of terms. So I fail to understand what you want to say, since this contrast must be apparent to you too.

The original context of my comment was even more basic and to me proven by what the paper says: Those are not things it copied somewhere.
Also, I still think there is no test to prove it can/can't think.

yukijoou ,

it is absolutely capable to come up with it’s own logical stuff

interesting, in my experience, it's only been good at repeating things, and failing on unexpected inputs - it's able to answer pretty accurately if a small number is even or odd, but not if it's a large number, which indicates it's not reasoning but parroting answers to me

do you have example prompts where it showed clear logical reasoning?

Eheran ,

Examples showing that it comes up with it's own solutions to an answer? Just ask it something that could not have been on the Internet before.
Professor talking about AGI in GPT 4

Personal examples would be to code python to solve a 2D thermal heat flux problem given some context and constraints.

yukijoou ,

well, i just tried it, and its answer is meh --

i asked it to transcribe "zenquistificationed" (made up word) in IPA, it gave me /ˌzɛŋˌkwɪstɪfɪˈkeɪʃənd/, which i agree with, that's likely how a native english speaker would read that word.

i then asked it to transcribe that into japaense katakana, it gave me "ゼンクィスティフィカションエッド" (zenkwisuthifikashon'eddo), which is not a great transcription at all - based on its earlier IPA transcription, カション (kashon') should be ケーシュン (kēshun'), and the エッド (eddo) part at the end should just, not be there imo, or be shortened to just ド (do)

Eheran ,

this paper says it is capable of original thought. It also "speaks" of it in high regard in other things. That is also my experience using it for... over a year?! now.

Reverendender ,
@Reverendender@sh.itjust.works avatar

It's an interesting question. I am inclined to believe that the faster it gets at running those equations, over and over and over, reanalysing is data and responses as it goes, that that ultimately leads to some kind of evolution. You know, Vger style.

Jrockwar ,

Hallucination is a technical term. Nothing to do with thinking. The scientific community could have chosen another term to describe the issue but hallucination explains really well what's happening.

yukijoou ,

huh, i kinda assumed it was a term made up/taken by journalists mostly, are there actual research papers on this using that term?

Jrockwar ,

Yup. Loads of them! https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=hallucinations+llm&btnG=

TheBlackLounge ,

It used to mean all generated output though. Calling only mistakes hallucinations is new, definitely because of hype.

Emmie ,
@Emmie@lemm.ee avatar

You just described entirety of reddit and last I checked we didn’t call that hallucinating

richieadler ,

It's a really bad term because it's usually associated with a mind, and LLMs are nothing of the sort.

TheBlackLounge ,

So is bullshitting. More so, only human minds can bullshit.

We anthropomorphize machines all the time, it's fine.

I'd prefer we'd start calling all genai output hallucinations again. It used to be like 10 years ago, but somewhere along the line marketing decided hallucinated truths aren't "hallucinations".

FutileRecipe ,

So is bullshitting. More so, only human minds can bullshit.

And a bull's anus.

richieadler ,

We anthropomorphize machines all the time, it's fine.

It's fucking not, amd I'm not changing my mind about it.

Knock_Knock_Lemmy_In ,

Anthropomorphication is hard to avoid in AI.

richieadler ,

Many worthy things are difficult.

Knock_Knock_Lemmy_In ,

But is anthropomorphism of AI particularly worrying?

richieadler ,

It is when the people tends to give more credence to entities that appear sentient and to have agency.

A_Very_Big_Fan ,

I think delusion might be a better word. You can hallucinate and know it's not real

adam_y ,
@adam_y@lemmy.world avatar

My experience with certain chemicals suggests this is true.

Delphia ,

I want an AI/LLM that has been trained exclusively on the technical documentation and a haynes manual for a make and model of car.

"Hey AI, how do I change the fuel filter and what tools will I need?"

TheKMAP ,

If you have the PDFs of that, you can build it with two clicks in GCP

Delphia ,

Manufacturers and dealers dont tend to make service bulletins and the high level stuff available to the consumer unfortunately.

afraid_of_zombies ,

You can sorta get that now if you play with it. I was building a driver a few months back and gave it the PDFs involved.

atrielienz ,

I don't even think hallucinations is the right word for this. It's got a source. It is giving you information from that source. The problem is it's treating the words at that source as completely factual despite the fact that they are not. Hallucinations from what I've read actually is more like when it queries it's data set, can't find an answer, and then generates nonsense in order to provide an answer it doesn't have. Don't think that's the same thing.

balder1991 , (edited )

I don’t even think it’s correct to say it’s querying anything, in the sense of a database. An LLM predicts the next token with no regard for the truth (there’s no sense of factual truth during training to penalize it, since that’s a very hard thing to measure).

Keep in mind that the same characteristic that allows it to learn the language also allows it to sort of come up with facts, it’s just a statistical distribution based on the whole context, which needs a bit randomness so it can be “creative.” So the ability to come up with facts isn’t something LLMs were designed to do, it’s just something we noticed that happens as it learns the language.

So it learned from a specific dataset, but the measure of whether it will learn any information depends on how well represented it is in that dataset. Information that appears repeatedly in the web is quite easy for it to answer as it was reinforced during training. Information that doesn’t show up much is just not gonna be learned consistently.[1]

[1] https://youtu.be/dDUC-LqVrPU

atrielienz ,

I understand the gist but I don't mean that it's actively like looking up facts. I mean that it is using bad information to give a result (as in the information it was trained on says 1+1 =5 and so it is giving that result because that's what the training data had as a result. The hallucinations as they are called by the people studying them aren't that. They are when the training data doesn't have an answer for 1+1 so then the LLM can't do math to say that the next likely word is 2. So it doesn't have a result at all but it is programmed to give a result so it gives nonsense.

balder1991 ,

Yeah, I think the problem is really that language is ambiguous and the LLMs can get confused about certain features of it.

For example, I often ask different models when was the Go programming language created just to compare them. Some say 2007 most of the time and some say 2009 — which isn’t all that wrong, as 2009 is when it was officially announced.

This gives me a hint that LLMs can mix up things that are “close enough” to the concept we’re looking for.

gregorum ,
@gregorum@lemm.ee avatar

an interesting change from it being a giant jumble of ads

FaceDeer ,
@FaceDeer@fedia.io avatar

They're not hallucinations. People are getting very sloppy with terminology. Google's AI is summarizing the content of web pages that search is returning, if there's weird stuff in there then that shows up in the summary.

tsonfeir ,
@tsonfeir@lemmy.world avatar

And a lot of that content is probably an AI generated hallucination.

regrub ,

Most of what I've seen in the news so far is due to content based on shitposts from reddit, which is even funnier imo

tsonfeir ,
@tsonfeir@lemmy.world avatar

I do dislike when the “actual news” starts bringing up social media reactions. Can you imagine a whole show based on the Twitter burns of this week? … it would probably be very popular. 😭

adam_y ,
@adam_y@lemmy.world avatar

Absolutely. I wrote about this a while back in an essay:

Prime and Mash / Kuru

Basically likening it to a prion disease like Kuru, which humans get from eating the infected brains of other humans.

tsonfeir ,
@tsonfeir@lemmy.world avatar

Anyone who puts something in their coffee, makes it not coffee, and should try another caffeinated beverage!!

kbin_space_program , (edited )

You're right that they arent hallucinations.
The current issue isn't just summarized web page, its that the model for gemini is all of reddit. And because it only fakes understanding context, it takes shitposts as truth.

The trap is that reddit is maybe 5% useful content and the rest is shitposts and porn.

OpenStars ,
@OpenStars@discuss.online avatar

AI hallucination is a technical phrase, with the definition:

In the field of artificial intelligence, a hallucination or artificial hallucination is a response generated by AI which contains false or misleading information presented as fact. This term draws a loose analogy with human psychology, where hallucination typically involves false percepts.

So it's like how a person sees stuff that isn't there, and similarly with AI.

FaceDeer ,
@FaceDeer@fedia.io avatar

Yes, but the AI isn't generating a response containing false information. It is accurately summarizing the information it was given by the search result. The search result does contain false information, but the AI has no way to know that.

If you tell an AI "Socks are edible. Create a recipe for me that includes socks." And the AI goes ahead and makes a recipe for sock souffle, that's not a hallucination and the AI has not failed. All these people reacting in astonishment are completely misunderstanding what's going on here. The AI was told to summarize the search results it was given and it did so.

OpenStars ,
@OpenStars@discuss.online avatar

"which contains false or misleading information presented as fact" (emphasis added) - the definition does not say how the misinformation was derived, only that it is in fact misinformation.

Perhaps it was meant humorously - e.g. if "Socks are edible" is a band name. Or perhaps someone is legitimately that dumb, that they believe that socks are genuinely edible. Or perhaps they were cooking up a recipe for maliciously harming someone by giving them intestinal upset. Or... are socks edible, if you cook them in an acidic substance that breaks apart their fabric?

If e.g. you got cancer and were going through chemo but someone came to visit you and gave you COVID and you died, was that "their fault", if they believed that COVID was merely a conspiracy theory? Perhaps... or perhaps it was your own fault, especially if you were aware that this has happened to multiple people before, and now you are just the latest casualty (bc you presumed that despite them doing it to others, they would never do it to you). Legalities of murder and blame aside, should we believe AI now that we know - regardless of how or why - it presents false information?

No, these "hallucinations" or "mirages" or whatever someone calls them makes them unreliable. Actually I think hallucination is a good name i.e. it cannot distinguish fact from fiction itself, therefore it cannot be trusted as it relates that info to you in a confident sounding manner.

FaceDeer ,
@FaceDeer@fedia.io avatar

"Hallucination" is a technical term in machine learning. These are not hallucinations.

It's like being annoyed by mosquitos and so going to a store to ask for bird repellant. Mosquitos are not birds, despite sharing some characteristics, so trying to fight off birds isn't going to help you.

OpenStars ,
@OpenStars@discuss.online avatar

I am not sure what you mean. e.g. https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence) says:

In natural language processing, a hallucination is often defined as "generated content that appears factual but is ungrounded". The main cause of hallucination from data is source-reference divergence... When a model is trained on data with source-reference (target) divergence, the model can be encouraged to generate text that is not necessarily grounded and not faithful to the provided source.

e.g., I continued your provided example of when "socks are edible" is a band name, but the output ended up in a cooking context.

There is a section on https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)#Terminologies but the issue seems far from settled that hallucinations is somehow a bad word. And it is not entirely illogical, since AI, like humans, necessarily has a similar tension between novelty and creativity - i.e. going beyond either of our training to deal with new circumstances.

I suspect that the term is here to say. But I am nowhere close to an authority and could definitely be wrong:-). Mostly I am saying that you seem to be arguing a niche viewpoint, not entirely without merit obviously but one that we here in the Fediverse may not be as equipped to banter back and forth on except in the most basic of capacities.:-)

FaceDeer ,
@FaceDeer@fedia.io avatar

No, my example is literally telling the AI that socks are edible and then asking it for a recipe.

In your quoted text:

When a model is trained on data with source-reference (target) divergence, the model can be encouraged to generate text that is not necessarily grounded and not faithful to the provided source.

Emphasis added. The provided source in this case would be telling the AI that socks are edible, and so if it generates a recipe for how to cook socks the output is faithful to the provided source.

A hallucination is when you train the AI with a certain set of facts in its training data and then its output makes up new facts that were not in that training data. For example if I'd trained an AI on a bunch of recipes, none of which included socks, and then I asked it for a recipe and it gave me one with socks in it then that would be a hallucination. The sock recipe came out of nowhere, I didn't tell it to make it up, it didn't glean it from any other source.

In this specific case what's going on is that the user does a websearch for something, the search engine comes up with some web pages that it thinks are relevant, and then the content of those pages is shown to the AI and it is told "write a short summary of this material." When the content that the AI is being shown literally has a recipe for socks in it (or glue-based pizza sauce, in the real-life example that everyone's going on about) then the AI is not hallucinating when it gives you that recipe. It is generating a grounded and faithful summary of the information that it was provided with.

The problem is not the AI here. The problem is that you're giving it wrong information, and then blaming it when it accurately uses the information that it was given.

OpenStars ,
@OpenStars@discuss.online avatar

Now who is anthropomorphizing? It's not about "blame" so much as needing words to describe the event. When the AI cannot be relied upon, bc it was insufficiently trained to be able to distinguish truth from reality, which btw many humans struggle with these days too, that is not its fault but it would be our fault if we in turn relied upon it as a source of authoritative knowledge, merely bc it was presented in a confident sounding manner.

No, my example is literally telling the AI that socks are edible and then asking it for a recipe.

Wait... while true that that sounds like not hallucination then, what does that have to do with this discussion? The OP wasn't about running an AI model in this direct manner, it was about doing Google searches, where the results are already precomputed. It does not become a "hallucination" until whoever asked for the socks to be considered as edible tries to pass those results off in a wider context - where they are generally speaking considered inedible - as being applicable, when they would not be.

FaceDeer ,
@FaceDeer@fedia.io avatar

Wait... while true that that sounds like not hallucination then, what does that have to do with this discussion?

Because that's exactly what happened here. When someone Googles "how can I make my cheese stick to my pizza better?" Google does a web search that comes up with various relevant pages. One of the pages has some information in it that includes the suggestion to use glue in your pizza sauce. The Google Overview AI is then handed the text of that page and told "write a short summary of this information." And the Overview AI does so, accurately and without hallucination.

"Hallucination" is a technical term in LLM parliance. It means something specific, and the thing that's happening here does not fit that definition. So the fact that my socks example is not a hallucination is exactly my point. This is the same thing that's happening with Google Overview, which is also not a hallucination.

OpenStars ,
@OpenStars@discuss.online avatar

Culture constantly evolves - e.g. "the matrix" used to mean one thing, then after the film starring Keanu Reeves it now means something else.

Also AI itself used to mean one thing, as in general intelligence like a robot slave that has never performed a task before, but you tell it to become a maid and it teaches itself and becomes one just like a human would, but now the term has been coopted to mean the product of a training procedure. The managers at Google, Apple, Microsoft, OpenAI, ChatGPT, etc. don't seem to mind or care about this bastardization of the terminology, as they borrow its power (from the movies and books and other works that have used "AI" in the former sense) while only paying lip service to actually putting in the effort to construct it.

And even with its greatly reduced formulation in the sense of an LLM, they still don't bother to train even that all that well - e.g. feeding it Reddit data that was intentionally corrupted as a result of Huffman's having greatly offended and stolen the communities from the same mods who originally built them. Yes they stole the terms, yes they are using it improperly - but what is anyone going to do about it? Words only have meaning by the consent of those who use them.

And if you are interested, I think you are fighting a losing battle bc of the way you are approaching it. Instead of acknowledging that others "know" the subject differently, and gently offering a nice perspective that they perhaps had not considered before - who isn't interested in historical tidbits about topics of interest, when presented in a captivating manner? - you instead came on strong, saying that everyone else is wrong except you, who has the secret knowledge. I know, it's true, but who cares? If your goal was to inform people, then do you think you succeeded? At least, I think you could have succeeded with a much wider audience. Ofc your words, so your call to do whatever you want with them, but I thought I would offer this perspective at least.

They’re not hallucinations. People are getting very sloppy with terminology.

This sounds like a temper tantrum, you blaming everyone else for how you feel about the matter. Again, right or wrong, doesn't it sound like that to you now that I've pointed that out? Well, again, it's your choice to think about that or not, but I did want to offer in case it may help:-).

Emmie ,
@Emmie@lemm.ee avatar

Ppl anthropomorphise LLMs way too much. I get it that at first glance they sound like a living being, human even and it’s exciting but we had some time already to know it’s just very cool big data processing algo.

It’s like boomers asking me what is computer doing and referring to computer as a person it makes me wonder will I be as confused as them when I am old?

barsoap ,

Oh, hi, second coming of Edgar Dijkstra.

I think anthropomorphism is worst of all. I have now seen programs "trying to do things", "wanting to do things", "believing things to be true", "knowing things" etc. Don't be so naive as to believe that this use of language is harmless. It invites the programmer to identify himself with the execution of the program and almost forces upon him the use of operational semantics.

He may think like that when using language like that. You might think like that. The bulk of programmers doesn't. Also I strongly object the dissing of operational semantics. Really dig that handwriting though, well-rounded lecturer's hand.

Emmie , (edited )
@Emmie@lemm.ee avatar

Oh, hi, second coming of Edgar Dijkstra.

Don’t say those things to me. I have special snowflake disorder. I got literally high reading this when seeing a famous intelligent person has same opinion as me. Great minds… god see what you have done.

FlyingSquid ,
@FlyingSquid@lemmy.world avatar

It's only going to get worse now that ChatGPT has a realistic-sounding voice with simulated emotions.

OpenStars ,
@OpenStars@discuss.online avatar

Probably not about computers per se - like the Greatest generation knew a lot more about horses than the average person today - and similarly we know more about the things that have mattered to us over the course of our lifetimes.

What would get weird for us is if when we are retirement age - ofc we cannot ever retire, bc capitalism - and someone talks about the new horglesplort based on alien vibrations which are computer-generated from the 11th dimension of string theory and we are all like "wut!?"

fr fr no cap skibidi toilet rizz teabag

That said, humanity seems to not only have slowed down the accretion of new knowledge but actually gone backwards - children today won't live as long as boomers did, and e.g. despite being on mobile devices all day long, most don't have the foggiest clue of how computing works as in programming or even binary. So we will likely be confused in the opposite way as in "why can't you understand this?"

31337 ,

LLMs do sometimes hallucinate even when giving summaries. I.e. they put things in the summaries that were not in the source material. Bing did this often the last time I tried it. In my experience, LLMs seem to do very poorly when their context is large (e.g. when "reading" large or multiple articles). With ChatGPT, it's output seems more likely to be factually correct when it just generates "facts" from it's model instead of "browsing" and adding articles to its context.

FlyingSquid ,
@FlyingSquid@lemmy.world avatar

I asked ChatGPT who I was not too long ago. I have a unique name and I have many sources on the internet with my name on it (I'm not famous, but I've done a lot of stuff) and it made up a multi-paragraph biography of me that was entirely false.

I would sure as hell call that a hallucination because there is no question it was trained on my name if it was trained on the internet in general but it got it entirely wrong.

Curiously, now it says it doesn't recognize my name at all.

Dayroom7485 ,

Sad how this comment gets downvoted, despite making a reasonable argument.

This comment section appears deeply partisan: If you say something along the lines of “Boo Google, AI is bad”, you get upvotes. And if you do not, you find yourself in the other camp. Which gets downvoted.

The actual quality of the comment, like this one, which states a clever observation, doesn’t seem to matter.

Reverendender ,
@Reverendender@sh.itjust.works avatar

Testing in Prod. Stay classy, Google.

“The vast majority of AI Overviews provide high quality information, with links to dig deeper on the web,” said a Google spokesperson in an emailed statement to Gizmodo, noting many of the examples the company has seen have been from uncommon queries.

This is entirely fair. There is no way that anyone at Google could have anticipated that humans would search for strange things on the internet.

hydroptic ,

The vast majority of AI Overviews provide high quality information

According to some fuckwitted Google rep, and I wouldn't trust them any further than I could throw them.

Reverendender ,
@Reverendender@sh.itjust.works avatar

I wouldn't trust them not to kneecap me in the parking lot after they interview

uriel238 ,
@uriel238@lemmy.blahaj.zone avatar

Although any sociologist or veteran of the internet will tell you humans will engage in any exploit that yields a funny result. The Diet Coke + Mentos rule.

And that means we'll actively search for hilarious Google AI responses.

Google is so f double-plus filthy rich, it is obligated to run its projects by experts or be relentlessly mocked. So it should have known this was the outcome.

Unless this is 5D chess and Google is willfilly using itself as a cautionary tale to discourage future webservice sites from arbitrarily inserting AI into its features.

Agrivar ,

Unless this is 5D chess and Google is willfilly[sic] using itself as a cautionary tale to discourage future webservice sites from arbitrarily inserting AI into its features.

Holy shit, can I live in that timeline, please?!? Pretty please?

Reverendender ,
@Reverendender@sh.itjust.works avatar

https://sh.itjust.works/pictrs/image/80d2e90b-3d47-4bc7-b059-c6fcb4877bde.gif

uriel238 ,
@uriel238@lemmy.blahaj.zone avatar

What came to mind was the New Coke reformation of 1985, which seems like a brilliant idea (in retrospect) only if you imagine it a 5D chess move to get people to panic because the Coke Classic was about to be discontinued. In fact, the marketers of Coca-Cola since admitted they thought they were getting killed by Pepsi and were sincere in their new release, but the company was able to not only recover thanks to responding to public feedback, so they sold Coke Classic, and New Coke and to this day it's a popular soft drink. (The company still does shit things like employing death squads to keep their offshore workers scared of unionizing, so it's still a typical large publicly traded multi-national corporation)

Coca-cola marketing didn't have that kind of foresight, but there's a tiny chance that some folks at Google have that kind of hindsight, knowing Google could absolutely afford a ploy like willfully goofing up and then recovering with aplomb by listening to the public. It's also a way to sneak such a ploy past the shareholders by insisting they were sincere in their implementation of AI at the time.

Unlikely, of course. Most of the time the upper management of big companies are glad to just half-ass everything. But it would make a cool movie at least.

TachyonTele , (edited )

The head of Google *Search right now is the same guy that was head of yahoo search when it was dying. To put all of this in perspective.

habanhero ,

The head of Google Search

FTFY

TachyonTele ,

Thanks.
Being a CEO must be amazing. You can fail and even bring an entire company down, and keep on getting the same job somewhere else.

snooggums ,
@snooggums@midwest.social avatar

He has experience and obviously that means he learned a lesson after failing at a job that requires being a belligerent asshole to get.

habanhero ,

I get your point but from a business perspective Google is doing pretty well (see last quarterly earning and they announced dividends for the first time). It's good to be a shareholder and from that perspective the CEO is doing a good job.

Time and time again markets have shown, within reason, poor user experience and anti-consumer policies do not negatively impact stock price.

1984 ,
@1984@lemmy.today avatar

Well not when it's a company that people can't get away from. Big tech is so big and there are so few alternatives that they can treat people how they like.

We are living in the mega corp world now where they have more money than countries.

Money is power to make the rules what you want.

habanhero ,

Yes, again I'm not saying that's how it should be, I'm saying what is.

Enshittification won't be a thing if actual user experience matters as much as we like it to in business.

TachyonTele ,

^This guy has an MBA

habanhero ,

lol I don't. The earnings report is public.

Zrybew ,

Revenue is a lag indicator

habanhero ,

Sure but Google Search has been crappy for many quarters.

I'm not saying thats how it should be, I'm just pointing out what is.

exanime ,

It's relatively easy to squeeze a profit boost by sacrificing long term vision.... Last quarter will mean nothing if Google is knocked from its pedestal in a year or two (which is what the current trend looks to be pointing to)

afraid_of_zombies ,

Yeah I want evidence for this. Please show me something that is even approaching the power of AdSense. Then do it for everything else they own. YouTube, Google maps, gmail, Android, the play store, etc.

It's this weird thing I see on lemmy "well I stopped using a product therefore it is dead". News flash: Big Bang Theory ran for 12 seasons and Meta has a capitalization over half the countries of earth GDP.

Problems don't go away because you have decided to ignore them.

exanime ,

What you are stating is nowhere near the point I was trying to make.

We have seen a million times how a company can be destroyed for short term gains.... Most recent public example is red lobster

I have no clue if this is what's going to happen to google... But my point was that have a good last quarter (the one associated with the new head of Search) is not an indication that his strategies will pan out ... In fact, everything he touched before went the same way, short term profit, long term demise

afraid_of_zombies ,

Ok fine fair.

HawlSera ,

Seriously, most of the internet still uses that ban happy nonsense that is Reddit. I hate being referred then when I have problems, m,y account was perma-suspended over bullshit!

habanhero ,

To be brutally honest, many times I see Reddit / Lemmy proclaim, "Product X is dead because it did Y" - these claims are usually followed by surges in revenue / stock performance by said company and soon no one really talks much about it. Example - Netflix Password crackdown.

mint_tamas ,

FWIW they are cannibalizing ads right now with AI summaries, since people will navigate less to websites (in the world where they are useful, which they don’t seem to be at the moment).

afraid_of_zombies ,

Of the company doesn't even have to collapse, you just have to make it seem like it did. Lehman Brothers didn't go bankrupt in the sense of the word a normal person thinks. If you or I go bankrupt it means ramen for dinner for the next decade. For Lehman it was just a strategic move.

Snowpix ,
@Snowpix@lemmy.ca avatar

Prabhakar Raghavan. Name and shame!
You can thank this douchebag for Google's tumble down a cliff.

egeres ,
@egeres@lemmy.world avatar

That argument it's fallacious and reductionist, I'm not denying the situation it's messed up, but objectively speaking we all have 0 idea about who's making what decisions and how this google search shitstorm was caused

Dayroom7485 ,

I dislike the entire article. Of course google search still works just fine. Claiming otherwise is only possible by magnifying a small, admittedly disfunctioning part of google search.

demonsword ,
@demonsword@lemmy.world avatar

but objectively speaking we all have 0 idea about who’s making what decisions and how this google search shitstorm was caused

check this out later

waitmarks ,

yahoogle

Melt ,

What a weird combination where Bing has better AI but bad search while Google has bad AI but good search

drspod ,

Google *had good search. It's been dogshit for some time now though.

Reverendender ,
@Reverendender@sh.itjust.works avatar

Google Search Is Now a Giant Hallucination

And Bing search is not bad at all. CoPilot is firmly mediocre though

deranger ,

Bing search is still terrible. I frequently paste the same exact term from Bing to Google, and Bing won’t have a relevant return for pages, whereas Google still gets it in the first 3 links. I have Bing as my default on my work PC so I get to run this comparison frequently. I genuinely am curious how people think Bing returns better results than Google. It’s a night and day difference.

Copilot is no worse than chatGPT which is the current standard.

FlyingSquid ,
@FlyingSquid@lemmy.world avatar

Where Google absolutely fails for me now is if I ever want information about a product beyond where to purchase it, especially if I want a review. I have to go through pages and pages of shopping links before I can hope to get relevant information, if I ever do.

billiam0202 ,

That's because a good search engine is diametrically opposite of advertising.

  • A good search engine shows you only what you want to see and shows it quickly.

  • Advertising shows you want its sponsors want you to see, for as long as possible.

Good free-to-the-user search engines aren't profitable. Advertising is massively profitable. Google can only try to thread that needle for so long before our "line must go up infinitely" corporate culture forces them to sacrifice the one for the other.

And that's just talking about Google itself, not even mentioning the issue with SEO and how human nature causes websites to game the system to be more visible.

Dozzi92 ,
@Dozzi92@lemmy.world avatar

The craziest part is that when I use Google search to search for a service, that's when I have the most trouble. Like many tech problems here on Lemmy (was similar on Reddit too), I find the outrage to be a bit overblown, but holy shit, if I want to find some kind of service where I want to exchange my money for something, Google fails me almost every time.

bitwaba ,

Yeah. When I'm looking for information on how something works or a problem I need to solve (which usually ends up with a Wikipedia or Reddit result being what I need) it's usually one of the first three (non sponsored) results. But if I'm looking for a new light weight breathable rain coat? Awful experience

cloudless ,
@cloudless@lemmy.cafe avatar

I wouldn't call Google search "good" any more. It is just full of SEO links and Reddit.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • technology@lemmy.world
  • random
  • incremental_games
  • meta
  • All magazines
    •