Redacted

Redacted , 1 hour ago

Cavity protection ain't gonna cut it where they're going.

Redacted , 9 days ago

🇺🇸 "Baadel a waader" 🇺🇸

Redacted , 11 days ago

Least contrarian Linux user.

Redacted , 9 days ago

Mmmmm fresh pasta.

For me it is very weird, no one introduced me formally to Lemmy(no one I knew run or heard of it), it felt like it was a legend. I never really got to know how good it was and always felt Reddit and Twitter were lacking, never really in control of your memes, never happy with my content, always downvoting stuff. The years went by and my curiosity only became larger as Reddit and Twitter experience was getting worse and worse. I already had experience shit posting and trolling on 4chan since my school days, so last year I signed up to Lemmy and posted my first meme. Next thing I know my feed is breathing again, the grass was definitely greener here. So I switched for both reasons.

Redacted , 19 days ago

Probably because it's down. I for one am surprised they have no failover for when Bing goes down.

Redacted , 28 days ago

✅ Math is hard

❌ This math is hard

Redacted , 3 months ago (edited 3 months ago)

This article, along with others covering the topic, seem to foster an air of mystery about machine learning which I find quite offputting.

Known as generalization, this is one of the most fundamental ideas in machine learning—and its greatest puzzle. Models learn to do a task—spot faces, translate sentences, avoid pedestrians—by training with a specific set of examples. Yet they can generalize, learning to do that task with examples they have not seen before.

Sounds a lot like Category Theory to me which is all about abstracting rules as far as possible to form associations between concepts. This would explain other phenomena discussed in the article.

Like, why can they learn language? I think this is very mysterious.

Potentially because language structures can be encoded as categories. Any possible concept including the whole of mathematics can be encoded as relationships between objects in Category Theory. For more info see this excellent video.

He thinks there could be a hidden mathematical pattern in language that large language models somehow come to exploit: “Pure speculation but why not?”

Sound familiar?

models could seemingly fail to learn a task and then all of a sudden just get it, as if a lightbulb had switched on.

Maybe there is a threshold probability of a positied association being correct and after enough iterations, the model flipped it to "true".

I'd prefer articles to discuss the underlying workings, even if speculative like the above, rather than perpetuating the "It's magic, no one knows." narrative. Too many people (especially here on Lemmy it has to be said) pick that up and run with it rather than thinking critically about the topic and formulating their own hypotheses.

Redacted , 3 months ago

Yep my sentiment entirely.

I had actually written a couple more paragraphs using weather models as an analogy akin to your quartz crystal example but deleted them to shorten my wall of text...

We have built up models which can predict what might happen to particular weather patterns over the next few days to a fair degree of accuracy. However, to get a 100% conclusive model we'd have to have information about every molecule in the atmosphere, which is just not practical when we have a good enough models to have an idea what is going on.

The same is true for any system of sufficient complexity.

Redacted , 3 months ago

We know Google Translate gets things wrong sometimes so I was just wondering if Russia means "Special" Military Operation in the same way the Americans mean "Special" Olympics?

Redacted , 4 months ago

I fully back your sentiment OP; you understand as much about the world as any LLM out there and don't let anyone suggest otherwise.

Signed, a "contrarian".

Redacted , 4 months ago

I believe OP is attempting to take on an army of straw men in the form of a poorly chosen meme template.

Redacted , 4 months ago (edited 4 months ago)

Whilst everything you linked is great research which demonstrates the vast capabilities of LLMs, none of it demonstrates understanding as most humans know it.

This argument always boils down to one's definition of the word "understanding". For me that word implies a degree of consciousness, for others, apparently not.

To quote GPT-4:

LLMs do not truly understand the meaning, context, or implications of the language they generate or process. They are more like sophisticated parrots that mimic human language, rather than intelligent agents that comprehend and communicate with humans. LLMs are impressive and useful tools, but they are not substitutes for human understanding.

Redacted , 4 months ago

Understanding is a human concept so attributing it to an algorithm is strange.

It can be done by taking a very shallow definition of the word but then we're just entering a debate about semantics.

Redacted , 4 months ago (edited 4 months ago)

Yes sorry probably shouldn't have used the word "human". It's a concept that we apply to living things that experience the world.

Animals certainly understand things but it's a sliding scale where we use human understanding as the benchmark.

My point stands though, to attribute it to an algorithm is strange.

Redacted , 4 months ago

Well it was a fun ruse while it lasted.

Redacted , 4 months ago

Yes you do unless you have a really reductionist view of the word "experience".

Besides, that article doesn't really support your statement, it just shows that a neural network can link words to pictures, which we know.

Redacted , 4 months ago

That last sentence you wrote exemplifies the reductionism I mentioned:

It does, by showing it can learn associations with just limited time from a human's perspective, it clearly experienced the world.

Nope that does not mean it experienced the world, that's the reductionist view. It's reductionist because you said it learnt from a human perspective, which it didn't. A human's perspective is much more than a camera and a microphone in a cot. And experience is much more than being able to link words to pictures.

In general, you (and others with a similar view) reduce complexity of words used to descibe conciousness like "understanding", "experience" and "perspective" so they no longer carry the weight they were intended to have. At this point you attribute them to neural networks which are just categorisation algorithms.

I don't think being alive is necessarily essential for understanding, I just can't think of any examples of non-living things that understand at present. I'd posit that there is something more we are yet to discover about consciousness and the inner workings of living brains that cannot be fully captured in the mathematics of neural networks as yet. Otherwise we'd have already solved the hard problem of consciousness.

I'm not trying to shift the goalposts, it's just difficult to convey concisely without writing a wall of text. Neither of the links you provided are actual evidence for your view because this isn't really a discussion that evidence can be provided for. It's really a philosophical one about the nature of understanding.

Redacted , 4 months ago

No one is moving goalposts, there is just a deeper meaning behind the word "understanding" than perhaps you recognise.

The concept of understanding is poorly defined which is where the confusion arises, but it is definitely not a direct synonym for pattern matching.

Redacted , 4 months ago

I agree, there is no formal definition for AGI so a bit silly to discuss that really. Funnily enough I inadvertantly wrote the nearest neighbour algorithm to model swarming behavour back when I was an undergrad and didn't even consider it rudimentary AI.

Can I ask what your take on the possibility of neural networks understanding what they are doing is?

Redacted , 4 months ago

Bringing physically or mentally disabled people into the discussion does not add or prove anything, I think we both agree they understand and experience the world as they are conscious beings.

This has, as usual, descended into a discussion about the word "understanding". We differ in that I actually do consider it mystical to some degree as it is poorly defined and implies some aspect of consciousness to myself and others.

Your definitions are remarkably vague and lack clear boundaries.

That's language for you I'm afraid, it's a tool to convey concepts that can easily be misinterpreted. As I've previously alluded to, this comes down to definitions and you can't really argue your point without reducing complexity of how living things experience the world.

I'm not overstating anything (it's difficult to overstate the complexities of the mind), but I can see how it could be interpreted that way given your propensity to oversimplify all aspects of a conscious being.

This is an argument from incredulity, repeatedly asserting that neural networks lack "true" understanding without any explanation or evidence. This is a personal belief disguised as a logical or philosophical claim. If a neural network can reliably connect images with their meanings, even for unseen examples, it demonstrates a level of understanding on its own terms.

The burden of proof here rests on your shoulders and my view is certainly not just a personal belief, it's the default scientific position. Repeating my point about the definition of "understanding" which you failed to counter does not make it an agrument from incredulity.

If you offer your definition of the word "understanding" I might be able to agree as long as it does not evoke human or even animal conscious experience. There's literally no evidence for that and as we know, extraordinary claims require extraordinary evidence.

Redacted , 4 months ago

Have you ever considered you might be, you know, wrong?

No sorry you're definitely 100% correct. You hold a well-reasoned, evidenced scientific opinion, you just haven't found the right node yet.

Perhaps a mental gymnastics node would suit sir better? One without all us laymen and tech bros clogging up the place.

Or you could create your own instance populated by AIs where you can debate them about the origins of consciousness until androids dream of electric sheep?

Redacted , 4 months ago (edited 4 months ago)

I'd appreciate it if you could share evidence to support these claims.

Which claims? I am making no claims other than AIs in their current form do not fully represent what most humans would define as a conscious experience of the world. They therefore do not understand concepts as most humans know it. My evidence for this is that the hard problem of consciousness is yet to be solved and we don't fully understand how living brains work. As stated previously, the burden of proof for anything further lies with yourself.

What definitions? Cite them.

The definition of how a conscious being experiences the world. Defining it is half the problem. There are no useful citations as you have entered the realm of philosophical debate which has no real answers, just debates about definitions.

Explain how I’m oversimplifying, don’t simply state that I’m doing it.

I already provided a precise example of your reductionist arguing methods. Are you even taking the time to read my responses or just arguing for the sake of not being wrong?

I've already provided my proof. I apologize if I missed it, but I haven't seen your proof yet. Show me the default scientific position.

You haven't provided any proof whatsoever because you can't. To convince me you'd have to provide compelling evidence of how consciousness arises within the mind and then demonstrate how that can be replicated in a neural network. If that existed it would be all over the news and the Nobel Prizes would be in the post.

If you have evidence to support your claims, I'd be happy to consider it. However, without any, I won't be returning to this discussion.

Again, I don't need evidence for my standpoint as it's the default scientific position and the burden of proof lies with yourself. It's like asking me to prove you didn't see a unicorn.

Redacted , 4 months ago

...or you might not.

It's fun to think about but we don't understand the brain enough to extrapolate AIs in their current form to sentience. Even your mention of "parts" of the mind are not clearly defined.

There are so many potential hidden variables. Sometimes I think people need reminding that the brain is the most complex thing in the universe, we don't full understand it yet and neural networks are just loosely based on the structure of neurons, not an exact replica.

Redacted , 4 months ago

You say maybe there's not much to understand about the brain but I entirely disagree, it's the most complex object in the known universe and we haven't discovered all of it's secrets yet.

Generating pictures from a vast database of training material is nowhere near comparable.

Redacted , 4 months ago (edited 4 months ago)

Possible, yes. It's also entirely possible there's interactions we are yet to discover.

I wouldn't claim it's unknowable. Just that there's little evidence so far to suggest any form of sentience could arise from current machine learning models.

That hypothesis is not verifiable at present as we don't know the ins and outs of how consciousness arises.

Then it would logically follow that all the other functions of a human brain are similarly "possible" if we train it right and add enough computing power and memory. Without ever knowing the secrets of the human brain. I'd expect the truth somewhere in the middle of those two perspectives.

Lots of things are possible, we use the scientific method to test them not speculative logical arguments.

Functions of the brain

These would need to be defined.

But that means it should also be reproducible by similar means.

Can't be sure of this... For example, what if quantum interactions are involved in brain activity? How does the grey matter in the brain affect the functioning of neurons? How do the heart/gut affect things? Do cells which aren't neurons provide any input? Does some aspect of consciousness arise from the very material the brain is made of?

As far as I know all the above are open questions and I'm sure there are many more. But the point is we can't suggest there is actually rudimentary consciousness in neural networks until we have pinned it down in living things first.

Redacted , 3 months ago (edited 3 months ago)

You obviously have hate issues

Says the person who starts chucking out insults the second they get downvoted.

From what I gather, anyone that disagrees with you is a tech bro with issues, which is quite pathetic to the point that it barely warrants a response but here goes...

I think I understand your viewpoint. You like playing around with AI models and have bought into the hype so much that you've completely failed to consider their limitations.

People do understand how they work; it's clever mathematics. The tech is amazing and will no doubt bring numerous positive applications for humanity, but there's no need to go around making outlandish claims like they understand or reason in the same way living beings do.

You consider intelligence to be nothing more than parroting which is, quite frankly, dangerous thinking and says a lot about your reductionist worldview.

You may redefine the word "understanding" and attribute it to an algorithm if you wish, but myself and others are allowed to disagree. No rigorous evidence currently exists that we can replicate any aspect of consciousness using a neural network alone.

You say pessimistic, I say realistic.

Redacted , 3 months ago

Once again not offering any sort of valid retort, just claiming anyone that disagrees with you doesn't understand the field.

I suggest you take a cursory look at how to argue in good faith, learn some maths and maybe look into how neural networks are developed. Then study some neuroscience and how much we comprehend the brain and maybe then we can resume the discussion.

Redacted , 3 months ago (edited 3 months ago)

You don't really have one lol. You've read too many pop-sci articles from AI proponents and haven't understood any of the underlying tech.

All your retorts boil down to copying my arguments because you seem to be incapable of original thought. Therefore it's not surprising you believe neural networks are approaching sentience and consider imitation to be the same as intelligence.

You seem to think there's something mystical about neural networks but there is not, just layers of complexity that are difficult for humans to unpick.

You argue like a religious zealot or Trump supporter because at this point it seems you don't understand basic logic or how the scientific method works.

Redacted , 3 months ago

You've just copied my arguments yet again.

Seek help, your projections are concerning.

Redacted , 3 months ago

And again...

Redacted , 4 months ago

They operate by weighting connections between patterns they identify in their training data. They then use statistics to predict outcomes.

I am not particularly surprised that the Othello models built up an internal model of the game as their training data were grid moves. Without loooking into it I'd assume the most efficient way of storing that information was in a grid format with specific nodes weighted to the successful moves. To me that's less impressive than the LLMs.

Redacted , 4 months ago

So somewhere in there I'd expect nodes connected to represent the Othello grid. They wouldn't necessarily be in a grid, just topologically the same graph.

Then I'd expect millions of other weighted connections to represent the moves within the grid including some weightings to prevent illegal moves. All based on mathematics and clever statistical analysis of the training data. If you want to refer to things as tokens then be my guest but it's all graphs.

If you think I'm getting closer to your point can you just explain it properly? I don't understand what you think a neural network model is or what you are trying to teach me with Pythag.

Redacted , 4 months ago

It wouldn't reverse engineer anything. It would start by weighting neurons based on it's training set of Pythagorean triples. Over time this would get tuned to represent Pythag in the form of mathematical graphs.

This is not "understanding" as most people would know it. More like a set of encoded rules.

Redacted , 4 months ago (edited 4 months ago)

Seems to me you are attempting to understand machine learning mathematics through articles.

That quote is not a retort to anything I said.

Look up Category Theory. It demonstrates how the laws of mathematics can be derived by forming logical categories. From that you should be able to imagine how a neural network could perform a similar task within its structure.

It is not understanding, just encoding to arrive at correct results.

Redacted , 4 months ago

You're being downvoted because you provide no tangible evidence for your opinion that human consciousness can be reduced to a graph that can be modelled by a neural network.

Addidtionally, you don't seem to respond to any of the replies you receive in good faith and reach for anecdotal evidence wherever possible.

I also personally don't like the appeal to authority permeating your posts. Just because someone who wants to secure more funding for their research has put out a blog post, it doesn't make it true in any scientific sense.

Redacted , 4 months ago (edited 4 months ago)

There you go arguing in bad faith again by putting words in my mouth and reducing the nuance of what was said.

You do know dissertations are articles and don't constitute any form or rigorous proof in and of themselves? Seems like you have a very rudimentary understanding of English, which might explain why you keep struggling with semantics. If that is so, I apologise because definitions are difficult when it comes to language, let alone ESL.

I didn't dispute that NNs can arrive at a theorem. I debate whether they truly understand the theorem they have encoded in their graphs as you claim.

This is a philosophical/semantical debate as to what "understanding" actually is because there's not really any evidence that they are any more than clever pattern recognition algorithms driven by mathematics.

Redacted , 4 months ago

Understanding as most people know it implies some kind of consciousness or sentience as others have alluded to here.

It's the whole point of your post.

Redacted , 4 months ago (edited 4 months ago)

No I'm not.

You're nearly there... The word "understanding" is the core premise of what the article claims to have found. If not for that, then the "research" doesn't really amount to much.

As has been mentioned, this then becomes a semantic/philosophical debate about what "understanding" actually means and a short Wikipedia or dictionary definition does not capture that discussion.

Redacted , 4 months ago (edited 4 months ago)

I've read the article and it's just clickbait which offers no new insights.

What was of interest in it to yourself specifically?

Redacted , 4 months ago (edited 4 months ago)

I question the value of this type of research altogether which is why I stopped following it as closely as yourself. I generally see them as an exercise in assigning labels to subsets of a complex system. However, I do see how the COT paper adds some value in designing more advanced LLMs.

You keep quoting research ad-verbum as if it's gospel so miss my point (and forms part of the apeal to authority I mentioned previously). It is entirely expected that neural networks would form connections outside of the training data (emergent capabilities). How else would they be of use? This article dresses up the research as some kind of groundbreaking discovery, which is what people take issue with.

If this article was entitled "Researchers find patterns in neural networks that might help make more effective ones" no one would have a problem with it, but also it would not be newsworthy.

I posit that Category Theory offers an explanation for these phenomena without having to delve into poorly defined terms like "understanding", "skills", "emergence" or Monty Python's Dead Parrot. I do so with no hot research topics at all or papers to hide behind, just decades old mathematics. Do you have an opinion on that?

Redacted , 4 months ago (edited 4 months ago)

Title of your post is literally "New Theory Suggests Chatbots Can Understand Text".

You also hinted at it with your Pythag analogy.

Redacted , 4 months ago (edited 4 months ago)

You posted the article rather than the research paper and had every chance of altering the headline before you posted it but didn't.

You questioned why you were downvoted so I offered an explanation.

Your attempts to form your own arguments often boil down to "no you".

So as I've said all along we just differ on our definitions of the term "understanding" and have devolved into a semantic exchange. You are now using a bee analogy but for a start that is a living thing not a mathematical model, another indication that you don't understand nuance. Secondly, again, it's about definitions. Bees don't understand the number zero in the middle of the number line but I'd agree they understand the concept of nothing as in "There is no food."

As you can clearly see from the other comments, most people interpret the word "understanding" differently from yourself and AI proponents. So I infer you are either not a native English speaker or are trying very hard to shoehorn your oversimplified definition in to support your worldview. I'm not sure which but your reductionist way of arguing is ridiculous as others have pointed out and full of logical fallacies which you don't seem to comprehend either.

Regarding what you said about Pythag, I agree and would expect it to outperform statistical analysis. That is due to the fact that it has arrived at and encoded the theorem within its graphs but I and many others do not define this as knowledge or understanding because they have other connotations to the majority of humans. It wouldn't for instance be able to tell you what a triangle is using that model alone.

I spot another apeal to authority... "Hinton said so and so..." It matters not. If Hinton said the sky is green you'd believe it as you barely think for yourself when others you consider more knowledgeable have stated something which may or may not be true. Might explain why you have such an affinity for AI...

Redacted , 4 months ago

Spot on.

Redacted , 4 months ago

To hijack your analogy its more akin to me stating a tree is a plant and you saying "So are these" pointing at a forest of plastic Christmas trees.

I'm pretty curious why you imagine you have so many downvotes?

Redacted , 4 months ago

Have you ever considered you might be the laypeople?

Equating a debate about the origin of understanding to antivaxxers...

You argue like a Trump supporter.

Redacted , 4 months ago

Lol indeed, just seen you moderate a Simulation Theory sub.

Congratulations, you have completed the tech evangelist starter pack.

Next thing you'll be telling me we don't have to worry about climate change because we'll just use carbon capture tech and failing that all board Daddy Elon's spaceship to teraform Mars.