Getting ROCM to work properly is like herding cats.
You need a custom implementation for the specific operating system, the driver version must be locked and compatible, especially with a Workstation / WRX card, the Pro drivers are especially prone to breaking, you need the specific dependencies to be compiled for your variant of HIPBlas, or zLUDA, if that doesn't work, you need ONNX transition graphs, but then find out PyTorch doesn't support ONNX unless it's 1.2.0 which breaks another dependency of X-Transformers, which then breaks because the version of HIPBlas is incompatible with that older version of Python and ..
Inhales
And THEN MAYBE it'll work at 85% of the speed of CUDA. If it doesn't crash first due to an arbitrary error such as CUDA_UNIMPEMENTED_FUNCTION_HALF
You get the picture. On Nvidia, it's click, open, CUDA working? Yes?, done. You don't spend 120 hours fucking around and recompiling for your specific usecase.
Also, you need a supported card. I have a potato going by the name RX 5500, not on the supported list. I have the choice between three rocm versions:
An age-old prebuilt, generally works, occasionally crashes the graphics driver, unrecoverably so... Linux tries to re-initialise everything but that fails, it needs a proper reset. I do need to tell it to pretend I have a different card.
A custom-built one, which I fished out of a docker image I found on the net because I can't be arsed to build that behemoth. It's dog-slow, due to using all generic code and no specialised kernels.
A newer prebuilt, any. Works fine for some, or should I say, very few workloads (mostly just BLAS stuff), otherwise it simply hangs. Presumably because they updated the kernels and now they're using instructions that my card doesn't have.
#1 is what I'm actually using. I can deal with a random crash every other day to every other week or so.
It really would not take much work for them to have a fourth version: One that's not "supported-supported" but "we're making sure this things runs": Current rocm code, use kernels you write for other cards if they happen to work, generic code otherwise.
Seriously, rocm is making me consider Intel cards. Price/performance is decent, plenty of VRAM (at least for its class), and apparently their API support is actually great. I don't need cuda or rocm after all what I need is pytorch.
So, AMD has started slapping the AI branding on to some of their products, but they haven’t leaned in to it quite as hard as Nvidia has. They’re still focusing on their core product line up and developing the actual advancements in chip design.
I think it's in the pipeline. AMD has bought Xilinx, which builds FPGAs and already had some AI specific cores in their processors. I believe they're developing that further and integrating it in their GPUs now.
I don't think it's a problem, more like a situation. You are not doing anything wrong or stupid, just interested in something new and promising and have the resources to pursue it. Good for you, may you find gold.
You can buy them new for somewhat reasonable prices. What people should really look at is used 1080ti's on ebay. They're going for less than $150 and still play plenty of games perfectly fine. It's the budget PC gaming deal of the century.
Lot of those games are also hot garbage. Baldur's Gate 3 may be the only standout title of late where you don't have to qualify what you like about it.
I think the recent layoffs in the industry also portend things hitting a wall; games aren't going to push limits as much as they used to. Combine that with the Steam Deck-likes becoming popular. Those could easily become the new baseline standard performance that games will target. If so, a 1080ti could be a very good card for a long time to come.
You're misunderstanding the issue. As much as "RTX OFF, RTX ON" is a meme, the RTX series of cards genuinely introduced improvements to rendering techniques that were previously impossible to pull-off with acceptable performance, and more and more games are making use of them.
Alan Wake 2 is a great example of this. The game runs like ass on 1080tis on low because the 1080ti is physically incapable of performing the kind of rendering instructions they're using without a massive performance hit. Meanwhile, the RTX 2000 series cards are perfectly capable of doing it. Digital Foundry's Alan Wake 2 review goes a bit more in depth about it, it's worth a watch.
If you aren't going to play anything that came out after 2023, you're probably going to be fine with a 1080ti, because it was a great card, but we're definitely hitting the point where technology is moving to different rendering standards that it doesn't handle as well.
"Instructions" is probably the wrong word here (I was mostly trying to dumb it down for people who aren't familiar with graphics rendering terminology).
The idea is that it allows offloading more work to the GPU in ways that are much better performance-wise. It just requires that the hardware actually support it, which is why you basically need an RTX card for Alan Wake 2 (or whichever AMD GPU supports Mesh Shaders, I'm not as familiar with their cards).
Ah, mesh shaders. Cool stuff. AMD retroactively added them to their old GPUs in drivers. I think same goes for Intel's post-Ivybridge GPUs(I think send opcode can throw primitives into 3d pipeline, if you are interested, you can go read docs). I guess Nvidia can do something similar.
And even if they don't have such straightforward way of implementing them, they probably(my guess, can be wrong) can be emulated in geometry shaders.
The Rog seems to be doing a little better, but not by much. They're both hitting sub 30fps at 720p.
My point is that if that kind of handheld hardware becomes typical, combined with the economic problems of continuing to make highly detailed games, then Alan Wake 2 is going to be an abberation. The industry could easily pull back on that, and I welcome it. The push for higher and higher detail has not resulted in good games.
I own a 1080ti and there was recently a massive update to Allan Wake 2 that made it more playable on pascal GPUs. Digital foundry did a video on it: http://youtu.be/t-3PkRbeO8A
I don’t know of any current game that can’t run at least 1080p30fps on 1080ti. But of course my knowledge is not exhaustive.
I wouldn’t expect every “next-gen” game to get the same treatment as Alan Wake 2 going forward. But we’re 4 years into the generation and there has probably been less than 10 games that were built to take full advantage of modern console hardware. My 1080ti has got a few more good years in it.
not in my country lol. getting used cards were already the norm before, for a while you could literally only get used ones for a good price on aliexpress.
and now our gvmnt imposed 100% tax on anything from china, so its really just not affordable.
You can still look for used ones locally either through hardwareswap or fb marketplace (unfortunately thats the best secondhand marketplace atm). Other options include liquidation companies, sometimes universities also have a big market (from both staff and students)
Its been pretty rough still though, good luck on your search
no such thing as hardwareswap in my country. our marketplaces are overpriced because we want to resell our shit with prices proportional to what we overpaid for it.
importing used shit is cheaper because the value of, say, an rx580 is much lower outside the country.
Yea lots of the secondhand tech market exists purely in the US. The market outside of there and a couple other countries (uk, ca, de) dont really exist afaik
Meanwhile I dont think I have played more than 30minutes on my ps5 this year and its june, and I have definitely not played any minutes on the 1080 sitting in my PC...
Oh fuck scratch that I may have played about 2 hours of Dune Spice Wars
Honestly, I don't have an opinion on it, it didnt capture me like old world did last year, so probably not as good or I am just preferring more slow, thought out turn based stuff.
But mostly I am just kinda over gaming as a whole, I realized it's mostly cheap dopamine chasing for me and I don't really enjoy it.
I mean if LLM/Diffusion type AI is a dead-end and the extra investment happening now doesn't lead anywhere beyond that. Yes, likely the bubble will burst.
But, this kind of investment could create something else. We'll see. I'm 50/50 on the potential of it myself. I think it's more likely a lot of loud talking con artists will soak up all the investment and deliver nothing.
It's looking like a dead end. The content that can be fed into the big LLMs has already been done. New stuff is a combination of actual humans and stuff generated by LLMs. It then runs into an ouroboros problem where it just eats its own input.
Yeah, I was thinking more if there's either an evolutionary improvement or revolutionary (or some movement toward AGI). For me it's better if not, so I get to keep my job for a few more years. But, my general feeling is with the cash injection, there's some chance of a breakthrough.
I mostly agree, with the caveat that 99% of AI usage today just stupid gimmicks and very few people or companies are actually using what LLMs offer effectively.
It kind of feels like when schools got sold those Smart Whiteboards that were supposed to revolutionize teaching in the classroom, only to realize the issue wasn't the tech, but the fact that the teachers all refused to learn and adapt and let the things gather dust.
I think modern LLMs should be used almost exclusively as an assistive tool to help empower a human worker further, but everyone seems to want an AI that you can just tell 'do the thing' and have it spit out a finalized output. We are very far from that stage in my opinion, and as you stated LLM tech is unlikely to get us there without some sort of major paradigm shift.
To be fair, electronic whiteboards are some of the jankiest piles of trash I've ever had to use. I swear to God you need to re-calibrate them every 5 minutes.
bubbles have nothing to do with technology, the tech is just a tool to build the hype. The bubble will burst regardless of the success of the tech at most success will slightly delay the burst, because what is bursting isnt the tech its the financial structures around it.
See Sun Microsystems after the .com bubble burst. They produced a lot of the servers that .com companies were using at the time. Shriveled up after and were eventually absorbed by Oracle.
Why did Oracle survive the same time? Because they latched onto a traditional Fortune 500 market and never let go down to this day.
I doubt it. Regardless of the current stage of machine learning, everyone is now tuned in and pushing the tech. Even if LLMs turn out to be mostly a dead end, everyone investing in ML means that the ability to do LOTS of floating point math very quickly without the heaviness of CPU operations isn’t going away any time soon. Which means nVidia is sitting pretty.
As far as I understand, the GPUs that LLMs use aren't exactly interchangeable with your regular GPU. Also, no one needs that many GPUs for any traditional use cases.
Worst one is probably Apple. They just announced "Apple Intelligence" which is just ChatGTP whose largest shareholder is Microsoft. Figure that one out.
Well, most of the requests are handled on device with their own models. If it’s going to ChatGPT for something it will ask for permission and then use ChatGPT.
So the Apple Intelligence isn’t all ChatGPT. I think this deserves to be mentioned as a lot of the processing will be on device.
Also, I believe part of the deal is ChatGPT can save nothing and Apple are anonymising the requests too.
Is this conjecture or can you provide some further reading, in the interest of not spreading misinformation.
Edit: I decided to read the info from Apple.
With Private Cloud Compute, Apple sets a new standard for privacy in AI, with the ability to flex and scale computational capacity between on-device processing, and larger, server-based models that run on dedicated Apple silicon servers. When requests are routed to Private Cloud Compute, data is not stored or made accessible to Apple and is only used to fulfill the user’s requests, and independent experts can verify this privacy.
Additionally, access to ChatGPT is integrated into Siri and systemwide Writing Tools across Apple’s platforms, allowing users to access its expertise — as well as its image- and document-understanding capabilities — without needing to jump between tools.
Say what you will about Apple, but privacy isn’t a concern for me. Perhaps, some independent experts will verify this in time.
Well, most of the requests are handled on device with their own models. If it’s going to ChatGPT for something it will ask for permission and then use ChatGPT.
I feel I was pretty explicit in explaining how some requests will go to ChatGPT.
Do you think that if you enter into a contract with a company like Apple they’ll just be like, aww shit they weren’t supposed to do that. Anyway let’s carry on.
No. This would open OpenAi up to potential lawsuits.
Even if they did save stuff. It gets anonymised by Apple before even being sent to ChatGPT servers.
I don’t want my comments here to be received as shilling Apple, more that I want them to based on actual information that is provided and not opinion pieces.
The fact is, if they were to caught saving data then Apple would just end the contract. Is it worth it for them to lose out on that cash, for the sake of using it. When they can just use all the other sources where they are allowed to do that.
Anyway, I don’t care what anonymised data they may or may not save. It won’t be tied to me.
Edit: Do you have some information on this existing lawsuits and the contracts they broke?
There's kind of a difference between "we scraped the internet and decided to use copyrighted content anyways because we decided to interpret copyright law as not being applicable to the content we generate using copyrighted content" (omegalul) and "we explicitly agreed to a legally-binding contract with Apple stating we won't do that".
That's just not true. Most requests are handled on-device. If the system decides a request should go to ChatGPT, the user is promped to agree and no data is stored on OpenAI's servers. Plus, all of this is opt-in.
I think there’s a larger picture at play here that is being missed.
Getting the weather is a standard feature for years now. Nothing AI about it.
What is “AI” is, Hey Siri, what is the weather at my daughter’s recital coming up?
The AI processing, calculated on-device if what they claim is true, is:
the determination of who your daughter is
What is a recital? An event? Are there any upcoming calendar events that match this concept?
Is the “daughter” associated with this event by description or invitation? Yes? OK, what’s the address?
Submit zip code of recital calendar event involving the kid to the weather API, and churn out a reply that includes all this information…
Well {Your phone contact name}, it looks like it will {remote weather response} during your {calendar event from phone} with {daughter from contacts} on {event date}.
That is the idea between on-device and cloud processing. The phone already has your contacts and calendar and does that work offline rather than educating an online server about your family, events and location, and requests the bare minimum from the internet, in this case nothing more than if you opened the weather app yourself and put in a zip code.
Voice processing is AI and was done by Apple servers. Previously, only the keyword "Hey Siri" was local. Onboard AI chips will allow this to be local. The actual queries will go to the servers. Phones do not have the power to run useful LLM locally- at least not with the near instantaneous response times phone users expect. A 56 Watt 128GB RAM M3 Max does around 8.5 tokens/second.
Perhaps this is why these features will only be available on iPhone 15 Pro/Max and newer? Gotta have those latest and greatest chips.
It will be fun to see how it all shakes out. If the AI can’t run most queries on the phone with all this advertising of local processing…there’ll be one hell of a lawsuit coming up.
EDIT: Finished looking for what I thought I remembered…
Additionally, Siri has been locally processed since iOS 15.
Forgive me, I’m no AI expert to fully compare the needed tokens per second measurement to relate to the average query Siri might handle, but I will say this:
Even in your article, only the largest model ran at 8/tps, others ran much faster, and none of these were optimized for a task, just benchmarking.
Would it be impossible for Apple to be running an optimized model specific to expected mobile tasks, and leverage their own hardware more efficiently than we can, to meet their needs?
I imagine they cut out most worldly knowledge etc/use a lightweight model, which is why there is still a need to link to ChatGPT or Apple for some requests, would this let them trim Siri down to perform well enough on phones for most requests? They also advertised launching AI on M1-2 chip devices, which are not M3-Max either…
Literally not what people are talking about. It's the "AI" part of the task that doesn't leave the device (unless it prompts to ask chat gpt). Not that it can magically gleam live info without making any request to the web...
Jeeze, fucking... get your shit straight, making me defend Apple... Fucking do better.
Not true. Most if not all requests are handled by apples own models on device or on their own servers. When it does use OpenAI you need to give it permission each time it does.
The US state department doesn't decide which countries own or control which territory, now does it? How exactly can you say territory you don't control (neither legally nor militarily) and likely will never control is part of your own country? Furthermore, why would the US risk ruining trade relations with China over unnecessarily pointing out reality, when it doesn't benefit the US to recognize Taiwanese independence?
I'll say it again: Why would countries risk ruining trade relations with China, one of the three most important trade powers internationally, over unnecessarily pointing out reality and thus contradicting China? And how can you seriously say territory a country doesn't control in any capacity at all theirs? Why do you think a majority of world powers are independently trading with Taiwan if Taiwan isn't independent from China?
Don't you think China would, you know, not be constantly complaining about not having control over Taiwan for the past few decades and making bluffs about invading if Taiwan were already part of China? That's a pretty obvious sign that "no, China doesn't own Taiwan in any capacity".
What you're doing here is called sophistry. Taiwan being part of China is a fact that's recognized by international law. It's really that simple. The reality is that China could remove US sponsored regime in the rogue province any time they want. However, they realize that it's much better to remove burgerland influence in a peaceful way, and that's what will happen. It's incredible how people have trouble grasping such basic things.
edit: I aboslutely love how utterly enraged lemmy radlibs get when faced with reality
What you're doing here is called sophistry. Taiwan being part of China is a fact that's recognized by international law.
Tell me you have no idea how the UN works without explicitly saying so. A majority of countries not recognizing Taiwan doesn't mean it's "international law" that Taiwan isn't independent.
It's really that simple. The reality is that China could remove US sponsored regime in the rogue province any time they want.
LMAO this is such a cope. Yeah I'm sure the extremely aggressive all-bark-no-bite and constant "you better not do <x diplomacy with Taiwan or military action in Taiwanese strait/South China Sea> again or we'll do something about it, I swear!" empire is suuuper capable of taking Taiwan. They know if they tried full-out war against the US or its allies (Taiwan), the US navy would cut off their international trade and turn their country upside down – it's why they're trying so hard (and failing) to seize full control of the South China Sea.
However, they realize that it's much better to remove burgerland influence in a peaceful way, and that's what will happen.
Again, absolute cope. They've been at it for over 75 years and haven't made any progress, considering Taiwanese have developed significantly more national identity and even more people in Taiwan support the country participating in international relations under the name "Taiwan" (80%) and consider themselves primarily Taiwanese (90%), and only 6% consider themselves more Chinese than Taiwanese (more people considered themselves primarily Chinese many decades ago but that has long since dwindled).
It's incredible how people have trouble grasping such basic things.
It's incredible how you have trouble grasping the situation and think China is going to "peacefully" absorb Taiwan when Taiwan is farther from China than ever in terms of national identity and international participation.
Several polls have indicated an increase in support of Taiwanese independence in the three decades after 1990. In a Taiwanese Public Opinion Foundation poll conducted in June 2020, 54% of respondents supported de jure independence for Taiwan, 23.4% preferred maintaining the status quo, 12.5% favored unification with China, and 10% did not hold any particular view on the matter. This represented the highest level of support for Taiwanese independence since the survey was first conducted in 1991. A later TPOF poll in 2022 showed similar results.
Tell me you have no idea how the UN works without explicitly saying so. A majority of countries not recognizing Taiwan doesn’t mean it’s “international law” that Taiwan isn’t independent.
I think you just told us that about yourself.
LMAO this is such a cope.
Proceeds to write a bunch of cope. 😂
. They know if they tried full-out war against the US or its allies (Taiwan), the US navy would cut off their international trade and turn their country upside down – it’s why they’re trying so hard (and failing) to seize full control of the South China Sea.
You losers can't even prop up Ukraine against Russia, and think you can take on China. The sheer delusion here. Burgerland economy would collapse overnight. Go check where all your shit comes from sometime. 😂
It’s incredible how you have trouble grasping the situation and think China is going to “peacefully” absorb Taiwan when Taiwan is farther from China than ever in terms of national identity and international participation.
You're like the living embodiment of the Dunning-Kruger effect.
I'll keep this short since you already seem extremely unhinged and half the stuff you wrote is basically empty insults.
You losers can't even prop up Ukraine against Russia, and think you can take on China.
Remind me how long that "10-day special operation" is taking again? Seriously, how can the "2nd best military in the world" falter so hard against their tiny neighbour with 1/4 of the population just because they got weapon donations from other countries? It shouldn't be that hard to counter right, I mean Russian military technology is allegedly so advanced and totally not stuck in the 80s. I would understand if it were half-way across the globe or something, but they're LITERALLY ON THEIR DOORSTEP. It's also concerning that China has repeatedly failed Russia when it comes to Ukraine and has caved into international pressure quite a few times, maybe it's because China also realizes that the war is completely embarrassing Russia?
The sheer delusion here. Burgerland economy would collapse overnight. Go check where all your shit comes from sometime. 😂
The US navy has a larger airforce than the entire Chinese airforce, and the US has a larger and more advanced air fleet than the next 5 countries (Russia, China, India, SK, Japan) combined, and invests 3x as much as China into the military (and that's what, 13% of the US' budget?). The US navy also has over 2x the displacement of the Chinese navy. Spending is DEFINITELY not a problem considering that.
I literally said that exact same thing in my original comment, it goes against your point lmao. The status quo is defacto independence and "Taiwan, not China". Notice how unification is by far the least popular response in what you linked, and has decreased in popularity significantly over decades. And of course, gaining independence eventually has increased in popularity over multiple decades. Is this part of China's grand plan, to make unification with them less popular over time?
Remind me how long that “10-day special operation” is taking again?
Yes, let me remind you that it's now publicly known that Ukraine was about to make a peace deal within the first couple months. Then the west tanked that deal, and started pumping weapons into Ukraine.
Seriously, how can the “2nd best military in the world” falter so hard against their tiny neighbour with 1/4 of the population just because they got weapon donations from other countries? It shouldn’t be that hard to counter right, I mean Russian military technology is allegedly so advanced and totally not stuck in the 80s. I would understand if it were half-way across the globe or something, but they’re LITERALLY ON THEIR DOORSTEP. It’s also concerning that China has repeatedly failed Russia when it comes to Ukraine and has caved into international pressure quite a few times, maybe it’s because China also realizes that the war is completely embarrassing Russia?
LMAO, China can just blockade the province and the economy there would crash overnight.
The only real metric to determine whether Taiwan is part of China is to ask the people who live there. And guess what, this is what they say:
Yeah, I agree, especially after all the NED sponsored propagandists are kicked out. Meanwhile, even despite US having a massive presence in Taiwanese media, the only reason DPP got in power was using first past the post mechanic.
Latest polling shows that vast majority of people want to maintain the status quo:
Do you honestly think a military blockade is the way to get people on the side of your cause?
I don't think that at all, and that's why China hasn't resorted to this option.
“After all the people I disagree with are got rid of” Do you honestly think that makes you the good guy? Removing political opponents is the methodology of authoritarians.
Entire books have been written explaining in great detail how media is used to manipulate public opinion, and here you are bleating about AuThoRiTarIanS. 🤡
You claim international law on one hand as if it means something and then threaten with the big stick with the other.
Not sure what the contradiction in your mind is of a country enforcing its laws within its borders. You'll have to enlighten us on this fascinating political theory of yours.
Put simply, you’re an asshole.
Put simply, you’re a cheap propagandist without a shred of intellectual honesty.
I’m not American nor have any loyalty to the Western hegemony. I’m an anarchist living in a country with its own independence movement attempting to get out from the boot of colonialism.
Anarchists are just edgy liberals, thanks for confirming that once again in this thread.
When you’re finished your work for the day propagandising, I hope you go home and think on your morality. Think on who you are as a person. Is defending Chinese bullying, violence, and threat really the best thing you can be doing with your life?
Hope you'll take your own advice, but we both know you won't. Clearly defending US colonialism is how you choose to spend your life. Peak anarchism right there.
China's puts about as much effort into developing their own shovel head manufacturing capability as we do fearmongering about a Chinese invasion of Taiwan, which is why they're rapidly closing the shovel head manufacturing gap.
Those things aren’t mutually exclusive. Yes, they are dumping massive resources into SMIC. Yes, they also want to maintain imperialism over Taiwan, and TSMC is a part of that. Some of it is fear-mongering sure, but China is consistently confrontational in the South China Sea and beyond. There’s a reason they enforce an abrasive naval presence there and continue to press against the Philippines.
they also want to maintain imperialism over Taiwan
Not to deny the realities of the tensions there, but liberals are relatively loose with term imperialism. There is a difference between an imperialist state like the US and an anti-imperialist — and until recently imperialized — state like China.
China is consistently confrontational in the South China Sea and beyond
You have an island governed by a democratically elected government, with a population that from what I remember mostly doesn't want to be assimilated into the PRC. The PRC taking it by force would, in my eyes, be rather imperialistic.
democratically elected? arguable and only for the last few decades at that. it was run as a brutal single party dictatorship backed by amerikkka until fairly recently. And last time i checked the vast majority of people in Taiwan want to maintain the status quo which is that Taiwan is part of China.
I think its largely the chip manufacturers, but ARM is still making money on licensing fees for Nvidia's new ai chip (with an integrated 72 core arm cpu) for example
ARM is in the perfect place where, if a company using their architecture succeeds, they get tons of money, and if the company fails, they lose nothing.
