How about, and run with me on this, Mozilla stops trying to be Microsoft and Google and instead just provides the cleanest, most barebones-yet-privacy-oriented browser? Will they ever have market dominance? No, and they never will even with AI tools. Fuck AI and what it's doing to the planet and fuck all of the capitalists enshittifying The Last Browser.
We need a new Foundation willing to develop a fork.
Decent models are huge; an average one requires 8GB to be kept in memory (better models requires something like 40 to 70 GB), and most currently available engines are extremely slow on a CPU and requires dedicated hardware (and even relatively powerful GPU requires a few seconds of "thinking" time). It is unlikely that these requirements will be easily squeezable in current computers, and more likely that dedicated hardware will be required.
I don't think any inference engines have actually been optimised to run on CPUs. You're stuck with 32-bit floats but OTOH that just means that you can do gigantic winograd transformations with the excess precision, needing far fewer fmuladds in total and CPUs are better at dealing with the memory access patterns that come with transforming the convolution. Most people have at least around 1TFLOP of compute in their CPU (e.g. a Ryzen 3600 has that much) that's not ever seeing the light of day. About a fifth of what an RX 570 has, it's a difference but not a magnitude and you can run SDXL with that kind of class of card (maybe not the 570 dunno about software support but a 5500 works, despite AMD's best efforts to cripple rocm).
Also from what I gather they're more or less doing summarybot for your browsing history, that's not a ChatGPT or Llama-style giant model you can talk with.
ONNX Runtime is actually decently well optimized to run on CPUs; even with large models. However, the simple truth is that there’s really no escaping that Billion+parameter models need to be quantized and even pruned heavily to fit in memory and not saturate the CPU cache so inferences/generations don’t take forever. That’s a reduction in accuracy, so the quality of the generations aren’t great.
There is a lot of really interesting research and development being done right now on smart quantization and pruning. Model serving technologies are improving rapidly too—paged attention is a really cool technique (for transformer based models) for effectively leveraging tensor core hardware—I don’t think that’s supported on CPU yet but it’s probably not that far off.
It’s a really active field and there’s just as much interest in running huge models on huge hardware as there is big models on small hardware. I recently heard of layerwise inference for CPUs; load each layer of the network to the CPU cache on demand. That’s typically a bottleneck operation on GPUs but CPU memoery so bloody fast that it might actually work fine. I haven’t played with it myself, or read the paper all that deeply so I can’t really comment more than it’s an interesting idea.
Sorry but has anyone in this thread actually tried running local LLMs on CPU? You can easily run a 7B model at varying levels of quantization (ie. 5 bit quantization) and get a generalized prompt-able LLM. Yeah, of course it's going to take ~4GB of RAM (which is mem-mapped and paged into memory), but you can easily fine tune smaller more specific models (like the translation one mentioned above) and have surprising intelligence at a fraction of the resources.
Take, for example, phi-2 which performs as well as 13B param models but with 2.7B params. Yeah, that's still going to take 1.5GB RAM which Firefox wouldn't reasonably ship, but many lighter weight specialized tasks could easily use something like a fine tuned 0.3B model with quantization.
Yes, I did. And yes, it is possible. It's terribly slow in comparison, making it less useful. It very quickly devolves into random mumbling or get stuck in weird loops. It also hogs resources that are actually used by other tasks you may be doing.
I mainly test dev AI solutions, and moving from 1B to 7B models made them vastly more pertinent. And moving from CPU implementation (Ryzen 7 3700X) to GPU (RTX 3080 Ti) made them fast enough to be used as quick completion and immediate suggestion without breaking workflow, in addition to freeing resources for IDE, building tools and the actual software being run, while running it on CPU had multi-seconds delay, which made this use case completely useless.
I don't really see it this way it's just marketing. Saying "all other browsers descend from big bad corporate Apple" is scary, saying "all other browsers descend from another open source project" is meh.
That's quite the bummer. But still. Saying that almost all browsers can trace their lineage to Apple and Webkit is technically correct, but it's just a half-truth. As Apple and Webkit were once based on KHTML.
I mean yes no kinda Konqueror simply accepted a bunch of downstream patches, including a name change.
...more or less. It could for a long time use all three of KHTML, WebKit (fork of KHTML) and QtWebEngine (Blink wrapped for Qt, that is, a fork of WebKit), they recently removed KHTML support because noone was updating it and it hadn't been the default for ages.
If they hadn't implemented multi-engine support in the past they probably would've switched over to "whatever Qt provides" right-out, it's KDE after all. Ultimately they're providing a desktop, not a web browser. Back in the days they did decide to roll their own instead of going with Firefox but it was never a "throw project resources at it" kind of situation, there were simply KDE people who felt like working on it. Web standards were a lot less involved back around the turn of the millennium, and also new and shiny. Back in the days people thought that HTML 4.01 Strict and XHMTL would be a thing that servers actually would start to output instead of the usual tagsoup.
If you're that kind of person right now I'll point you in the direction of Servo. No, Firefox doesn't use it and it's not a Mozilla project any more, Firefox only included (AFAIK) parallel CSS handling, the rest is still old Gecko.
uggggggggggh. I'm using Firefox because chrome is really going too far with it's manifest v3 garbage killing decent adblockere and Firefox is basically the only non chromium based option. Please for the love of everything that is holy. Just. Make. Your. Browser. Better. Don't need ai gimmicks. Definitely don't need to lay people off. You need to get back on track. Holy heck. This is the worst.
That's not the only use case. It could read a 400 page pdf locally and summarize it for you, answer questions and find which slide the data you want is on.
The use cases are only limited by how powerful the AI is
I'm starting to think FF is being deliberately run into the ground by the higher ups. It would be good to hear from some of the devs about their thoughts on all this.
The "upvote good, downvote bad" mentality needs to die. As others have said, the arrows are to promote/reduce visibility of content. Whether you agree with the content of the post should be irrelevant.
People have been saying some variant of this at least as far back as Slashdot in the late 90s. Nobody has come up with a viable way to change peoples habits.
Instead of fighting it, what can we do knowing that this is how it works?
New idea I've just come up with, we make it so, before upvoting or downvoting something, you have to press a button, and then wait at least a minute, or, better yet, solve a captcha, and then you don't even have to have accounts anymore and that takes about a minute. The only people upvoting or downvoting will be those who are really reflecting on what it is that they're doing, or the people who are really really committed and pissed off about something. I'm sure the latter won't happen like, ever.
I think maybe that's exactly how people are using it, it's just that most people aren't thinking "oh, well, this post made me a little mad or uncomfy, but I like the content and discussion that it's spurned, so I'll toss it an upvote". I think most people are more inclined to go "THIS POST MADE ME MAD! GRUG DOWNVOTE!". It doesn't even really not make sense, it would be kind of insane to spend like, even just a minute, thinking consciously about every single upvote or downvote you make, it would take a million years for anyone to ever upvote or downvote anything, and a lot of people would just not engage unless they were really committed, which doesn't necessarily map to their level of discernment, but might just instead map to how mad people could get over a given thing. Plenty of people could get mad enough about a thing to sit through a minute long wait period to downvote something.
Let me share some fun Mozilla facts about their previous CEO who has now stepped down to “executive chairwoman” last week.
She received 6.9 million dollars in 2022 and 5 million in 2021, 3 million in 2020.
Her replacement is an executive from AirBnB and eBay. We will find out how much both of these are earning in 2025 when they release their financial statements.
They fired 60 staff and are adding AI to their flagship program to earn more money.
Ok. Mozilla was spreading itself too thin, spending resources trying to compete with multiple products against established brands that were already way ahead of them. They needed to focus down onto their core product rather than frivolously cast about.
And AI is the technology of the future, despite all the whinging and griping by commenters on the subject. It's being incorporated into the other major browsers, it's a must-have if Firefox is to remain relevant. I'm sure you'll be able to turn it off in the settings if you don't want it and if you're really concerned about getting AI cooties there'll be niche forks that are compiled without it.
And the ever increasing CEO wages and hiring of AirBnB/eBay executive as CEO? Their previous CEOs salary alone could've covered everyone of those employees fired.
I agree with you that Mozilla is spreading itself too thin. And don't get me wrong, I love Firefox and am a long time user. But they do need to understand their user base better.
They aren't going to become a sustainable business by copying more popular browsers. It's their differences from the mainstream that make them appealing as an alternative in the first place. I already don't like them foisting Pocket on me, which 100% should have remained an extension. I don't like the fact that Google is their default search engine, which goes against all their privacy messaging. I understand the reason is money, but that's kind of the definition of being a sellout isn't it? Their core values should always come first.
Fact is, those employees weren't fired for any good reason other than to hop on the latest tech trend. It's this sort of corporate "profit before people" bullshit that will erode any goodwill that people still have towards Mozilla. I couldn't give a fuck about adding a stupid AI driven chatbot to Mozilla, and neither, I imagine, do many of their current users. Honestly, I think "AI" has ruined the internet in a lot of ways already. It's already had a massive negative impact on the quality of search results, across all major search engines, because of all the low quality llm content that has been produced already, and it's only going to get worse. And you can't trust a single thing that comes out of those models, so what is even the point of them?
As fair as I am aware, Mozilla so far is only thinking about integrating AI in relatively smart ways that leverage their limited resources well. (There were some rumours a while back about using ai locally to search your history and tabs, as well as (arguable if this counts as AI, but branding is everything) on device translation)
I think the obvious worry being alluded to is the reason they had 400m in cash due to their arrangement with Google. Their primary sustenance comes from an entity actively seeking their destruction.
And AI is the technology of the future, despite all the whinging and griping by commenters on the subject.
The entire discussion is to distract ourselves from the raw truth:
Fax machines are the technology of the future.
Fax machines will outlive us all. AI and VR will reach their heyday, then wane with years and be replaced. But whatever replaces them will sit quietly in the shadow of the everlasting Fax machine.
And AI is the technology of the future, despite all the whinging and griping by commenters on the subject.
Yeah because we've never seen tech fads before heralded as the next big thing. If I could roll my eyes any harder we could harness that for power generation.
And AI is the technology of the future, despite all the whinging and griping by commenters on the subject.
You have no idea, any more than the rest of us. Like, please tell me you understand "____ is the technology of the future" has been said more times than it's ever been true.
The idea of AI is a technology of the future, but what we have growing now is not AI, not really, and this iteration can be just as big a flop as any other technology of the moment.
LLMs are what everyone dunks on, and “image generators are coming for our jobs! Think of artists! It’s not real art if a cheating machine does it!” is also a common cry.
But do any of those people even know about the new class of antibiotics a neural network trained to find patterns in protein folding discovered? Do any of them know about the accuracy of diagnosis that IBM Watson was able to make in cases of rare cancers, even when doctors didn’t see it? What about changes in weather prediction accuracy? Novel suggestions in materials science?
We are mimicking neural patterns, similar to the way our own minds work, to achieve pattern recognition and even extrapolate from them. And yeah, right now we’re brute forcing it, and we’re not even entirely sure how these relationships develop. It’s in its infancy, and growing fast.
This is technology considered the holy grail of computing. We have been chasing this concept since the 1940s. There are a million sci-fi stories about it and there are a million more attempts to make it work before one really stuck.
And now we’re at the beginning of it being practical and you think we’re just gonna go “eh it’s a wet fart like the Virtual Boy. Oh well, let’s make some new phones or something”?
No. This is literally the technology of the future. Within your lifetime (assuming you live a reasonable while longer) there will come a point when you won’t be able to buy a CPU without some type of neural engine in it.
And yes, people will (and already are) do horrific shit with it. It will fuck over a large portion of the white collar economy; a portion of which were told to go into the careers they did because they’d be safe from automation. “Get a degree and you’ll be safe!” they told us! Now they tell us “you better work at two different targets to make that payment, should have studied a trade!”
So the reason for skepticism and animosity is almost certainly the fear of being replaced; but look at how far these AI models have come in the last month alone. We’re already in “this is changing the future” territory and those things are just getting started.
Dude. Take a chill in the bathtub and touch grass. AI is never taking my job, since it's physical labor since I removed myself from the computer industry 15 years ago. But as someone who studied AI and LISP (which was mired in the previous AI craze), it's not actually wrong to have animosity and be skeptical about the current AI. we're literally using the same techniques than we did 30 years ago. We've invented nothing new since the last AI fad. What is driving this craze is the brute force approach of massive parallel processing, not actual innovation.
There's been some minor refinement, so it's not exactly identical, but to use a metaphor... We've using more Lego bricks and different colours now to build our castles, but they're all still lego bricks. Nothing has fundamentally changed.
... and you should know by now that tech industry is funded by hype machine, so temper your expectations. Current machine learning techniques are limited and inefficient, it's not actually really a solvable problem with the current approach.
TLDR; LLMs are a super far cry from actually being "intelligent" and calling it AI is the equivalent of calling a wheeled electric self balance board a "Hoverboard".
This is technology considered the holy grail of computing.
This shit is just analog computing though, right? Like at it's base, we're just reproducing analog computation in a digital environment and then we're framing that in a million different ways, like we've been doing since the seventies. We've actually had this shit since the first computers, which were analog. The whole reason we moved to digital, though, is because the results were easier to break down, parse, and we had control over every step of the process to confirm it was correct, and it was going to be correct every time. A clearer sense of limitations and constraints, basically.
Now I'm not entirely against analog computing as a matter of fact, right, in fact I think it can be pretty cool if we recognize it for what it is, but at the same time I can't help but think that the level of hype around it is fucking insane. Primarily because it's not easily controllable or reproducible. Not in the sense that we're gonna somehow invent a rogue AI that will kill us all, or whatever garbage, but in the sense that, while you can get easily reproducible results (such is the nature of computation), it is very hard to control what the output is of a given neural network. You can process loads of information extremely quickly, but, like, what use is that if I don't know whether or not the solution is correct, or if it's just a kind of ballpark figure? That's the main issue.
Again, fine if we recognize it, but I don't think we're really close at all to just like, randomly inventing a rogue consciousness. We're not anywhere close to that, from what I've seen. We're still barely good at image recognition and generation in an actually complicated environment, and even then it's still pretty hard to get what it is that you specifically want, partially because the hype is driving so much development at this point, and the implementation is bunk and, again, kind of uncontrollable. Venture capital jumping down this thing's throat has partially blocked it's airway, as I see it. Still a useful technology, potentially, but a million stupid tech demos and image generators for nonsensical memes that we can flood everyone with is the dumbest shit imaginable, and even dumber than that is the level of venture capitalists I see that want to somehow monetize that.
And so I have to ask, right, if I want a robot to sort through the different colors of little plastic beads, right, do I get a large language model on that, or do I just run a pretty basic and more efficient algorithm that just narrows the parameter of beads to a certain color, as recorded by the camera, and then that's it? Do I want to translate a sentence with AI, or do I want to just manually run a straight word to word conversion that maybe changes based on a couple passes I'm gonna run at it to check whether or not it contextually makes sense with something like a markov chain? Trick question, they are both the same approach, AI has just done it in a way where I could apply a kind of broader paintbrush to the thing and get my results a little faster and with a little less thought even if I have less control over it.
Mozilla has long been the most ethical player in this space (while still producing SOTA ML). All of their datasets/models are open source and usually crowdsourced. Not to mention, their existing work is primarily in improving accessibility.
ALSO, the other half of this story is that Firefox is becoming the primary focus again. Everybody's freaking out about the AI stuff but that's because they're only reading the headlines. The programs they've shut down are things like Hubs (Mozilla's metaverse platform), the VPN, and the sensitive data scrubber (which was using a third party service anyway).
As a software developer I am huge supporter of Mozilla's developer initiatives from Manifest V2 implementation to MDN. But it's also important to be realistic Mozilla has long had major money problems, and not the kind that giving them more would fix.
I don't think this is a money making move. The previous CEO was absolutely overly focused on monetization and this move is a step away from that. I should've addressed this more explicitly in the above comment but even for the players who actively monetize, AI is a money incinerator.
The Lunduke shit again? The one that takes offense to money being donated to support "politics" i.e abortion rights?
Take a look at the other trash he posts on his reddit profile. That blog is not a trustworthy source, by any stretch, and it's sweetly ignoring that he's not looking at Mozilla's spending alone, but of 3 separate entities that exist under the umbrella of the Mozilla Foundation.
I don't know anything about him, but the criticism of them spending money on donating to other charities rather than focusing on making Mozilla's core projects sustainable IMO is correct.
The answers to both of those things depends very heavily on the details. I think focusing on their main products is a good thing, but adding AI sounds like one of those likely terrible decisions. We definitely need privacy friendly & open source based AI though, in all areas, so I hope this is Mozilla pushing for something sensible here.
You're right. Mozilla is the devil. Everyone go to the better option in Silicon Valley for web browsing....
...
...
...
...
Her replacement is an executive from AirBnB and eBay. We will find out how much both of these are earning in 2025 when they release their financial statements.
Can you tell me what they were doing at either of those companies, or what they've been doing at Mozilla since they were hired there? Have you done any actual research into this, at all, are you just assuming that because you saw two shitty companies on the resume, they must be a champion of those shitty companies?
I've been trying Arc browser that has a bunch of AI shoved in it and.... It's actually kinda nice. I think Firefox COULD possibly not fuck this up. Before you down vote me, I too believe that Firefox would be better off focusing on the core browser experience. And I really hope they have a good solution to AI being all cloud based right now. Like having a lightweight local model. This is why I was glad Arc was trying it, not Firefox.
