”However, if it is performance you are concerned about, “it’s important to note that GPUs still far outperform NPUs in terms of raw performance,” Jessop said, while NPUs are more power-efficient and better suited for running perpetually.”
Ok, so if you want to run your local LLM on your desktop, use your GPU. If you’re doing that on a laptop in a cafe, get a laptop with an NPU. If you don’t care about either, you don’t need to think about these AI PCs.
Or use a laptop with a GPU? An npu seems to just be slightly upgraded onboard graphics.
I will block the AI on any computer I use.
This is such a hilarious bit of branding nonsense. There is no such thing as “AI PCs”.
I mean, I technically own one, in that the branding says I do and it has a Copilot button, but… well, that’s definitely not why I purchased it and I don’t think I’ve used an “AI feature” on it. I’m not even opinionated against them, I have run local LLMs in my other computers, it’s just not a good application for the device I own that is specifically branded for “AI”.
The stupidity of it is that my “AI device” is an ARM device, and I absolutely love the things ARM Windows does that are actually useful. I pulled up my old x64 device that I used before I got this and man, the speed of Windows Hello, how much better it handles video streams, the efficiency… I’d never go back for a portable device at this point.
But the marketing says it’s “AI”, so once people start telling each other that “AI PCs” are bad and new AMD and Intel “AI” CPUs are released it’s anybody’s guess how the actually useful newer Windows ARM devices will fare.
I’m still hoping that the somewhat irrational anger towards “AI” stuff subsides so we can start talking about real features now, because man, this has been a frustrating generation to parse for portable Windows devices, and we still have Android, iOS and Mac devices coming down the pipe with similar branding nonsense.
I think this is referring to machines that come with APUs that have enough tensor (or whatever the equivalent is) cores in order to provide a certain baseline of “AI” operations per second, so Microsoft et al can rely on that for Recall etc without users thinking it slow.
Amusingly, the first generation of these didn’t really have enough, so they’re truly unwanted.
Yeah, my problem is that this was made to coincide with the Snapdragon Windows PCs, which are really good at a bunch of stuff and specifically not good at NPU performance, so the result of the “AI” branding ends up being really disappointing.
We could talk about all the other growing pains and the ways those devices were covered, but the obsessive focus on “AI” certainly didn’t help, as demonstrated by the bizarre reporting linked in the OP.
I’m still hoping that the somewhat irrational anger towards “AI” stuff subsides
I think this anger is linked to the irrational exuberance for “AI”.
Personally, I kinda hate AI. Not because of any sort of fear of job loss or anything like that. It’s because “AI” has been rolled out heavily in the Cybersecurity space, making my work life hell because of it. Models are only as good as their training and this means that any AI model which is going to spot anomalies in a network needs to spend a good amount of time being trained. However, what the vendors sell are touted as unsupervised models. They just need to spend a while on your network and they can automagically learn what “normal” is and then alert you on “abnormal”. This ignores the fact that you still need your analysts chasing false positives constantly from this black box. And that “black box” aspect is a major problem. You’ll get an AI/ML based alert with exactly fuck all in detail on why the alert triggered. If you’re lucky, you might get a couple log entries along with the alert, but nothing saying why those entries are suspicious.
I will grant that, there are a few cases where the “AI” in a product has worked. Mostly, it’s been in language processing. Heck, having an AI half-write a function for you in a tool you don’t use very often is quite nice. You almost always need to rework the results a bit, but it can get you started. But, my first question for any vendor talking about “AI Detections” is “how do we tune false positives?”. It’s just too big of a headache. And most of them try to downplay the need or dodge the question. Or, you have to babysit the model, effectively making it a supervised model. Which, fine. Just stop telling me how much time it’s going to save me, when I’m going to spend more time supervising the model than searching for threats in my environment. And, for fucks sake, design that shit to explain itself.
As for putting AI in my system. I can see a use case for language processing. Heck, I’d love to have the Star Trek style, “hello computer…” type stuff actually work worth a damn. Google and Siri are pretty close, though even those can be shit on toast when trying to do anything slightly complex. And having all that done locally, without having to send data “to the cloud” sounds great for privacy and security (until MS adds a keylogger as part of the OS). But, given how much time my GPU sits at or very near idle, I do wonder if the extra chip is worth the silicon or space.
In the end, I’m expecting this to go much the way TPM has. We’ll all end up with it in our systems, whether or not we know, care or use it. All because manufacturers just start soldering it on to everything. Maybe someone will find a good use for it eventually, distributed AI porn, maybe? But, like a lot of AI, it seems like a solution in search of a problem.
I’ve been extremely annoyed with all partisanship on this issue from the go. The absolutely insane range of promises, hyperbole and circlejerking among the AI techbros is absolutely insufferable. Meanwhile, the other side has been entirely content with getting a chance to repeat the dogpile they (deservedly) gave cryptobros and entirely unwilling to acknowledge any sort of nuance, often to the point of deliberate misinformation.
The only reason I’m not angrier at this is that, honestly, this whole thing is way less momentous than either side has claimed and I have bigger things to get mad about.
What are you referring to about Windows Hello?
Windows Hello existed long before AI became a trendy thing.
The instructions supported by NPU processors are designed for tasks like image recognition.
So, theoretically Hello might be quite a bit faster or more reliable (for a given time or energy budget) with an NPU.
The marketing is 100% drivel though.
Nothing to do with NPUs, as far as I know, it’s down to the dedicated video stream acceleration that ARM chips have carried for a while.
In practical terms, Windows Hello unlocks noticeably faster, just like a phone with facial recognition does. It also disproportionately handles video playback more efficiently, given the relative overall performance. Nothing new, either, it’s all stuff Apple got out of their own ARM transition, but way underrated in how much of a moment-to-moment difference it makes, particularly for stuff like the Surface Pro and other hybrids, where you want to handle it on battery and use it more like a tablet sometimes.
Just saying “built on AI” or whatever isn’t a convincing sales pitch. What can I actually do with AI that will improve my day to day life? Not a single advert or pitch has told me a single use case for this that applies to what anyone would use for a personal computer, and they’re too risky to buy for employees in a work environment unless you can afford to be the guinea pig for this unproven line of hardware (in the sense that I know a ThinkPad will last 10 years but I have no idea how long a copilot pc will last, how often I need to replace the battery or ram or anything else). I’m aware of tech, I know what these laptops are, but as far as I can see the market for them just does not exist and I don’t understand why anyone would think otherwise.
Have they tried forcing people to upgrade to AI PCs in order to receive security updates by checking to see if your PC is an AI PC? You know just to prove people really want AI PCs?
/s
