Just saying “built on AI” or whatever isn’t a convincing sales pitch. What can I actually do with AI that will improve my day to day life? Not a single advert or pitch has told me a single use case for this that applies to what anyone would use for a personal computer, and they’re too risky to buy for employees in a work environment unless you can afford to be the guinea pig for this unproven line of hardware (in the sense that I know a ThinkPad will last 10 years but I have no idea how long a copilot pc will last, how often I need to replace the battery or ram or anything else). I’m aware of tech, I know what these laptops are, but as far as I can see the market for them just does not exist and I don’t understand why anyone would think otherwise.
This is such a hilarious bit of branding nonsense. There is no such thing as “AI PCs”.
I mean, I technically own one, in that the branding says I do and it has a Copilot button, but… well, that’s definitely not why I purchased it and I don’t think I’ve used an “AI feature” on it. I’m not even opinionated against them, I have run local LLMs in my other computers, it’s just not a good application for the device I own that is specifically branded for “AI”.
The stupidity of it is that my “AI device” is an ARM device, and I absolutely love the things ARM Windows does that are actually useful. I pulled up my old x64 device that I used before I got this and man, the speed of Windows Hello, how much better it handles video streams, the efficiency… I’d never go back for a portable device at this point.
But the marketing says it’s “AI”, so once people start telling each other that “AI PCs” are bad and new AMD and Intel “AI” CPUs are released it’s anybody’s guess how the actually useful newer Windows ARM devices will fare.
I’m still hoping that the somewhat irrational anger towards “AI” stuff subsides so we can start talking about real features now, because man, this has been a frustrating generation to parse for portable Windows devices, and we still have Android, iOS and Mac devices coming down the pipe with similar branding nonsense.
I think this is referring to machines that come with APUs that have enough tensor (or whatever the equivalent is) cores in order to provide a certain baseline of “AI” operations per second, so Microsoft et al can rely on that for Recall etc without users thinking it slow.
Amusingly, the first generation of these didn’t really have enough, so they’re truly unwanted.
Yeah, my problem is that this was made to coincide with the Snapdragon Windows PCs, which are really good at a bunch of stuff and specifically not good at NPU performance, so the result of the “AI” branding ends up being really disappointing.
We could talk about all the other growing pains and the ways those devices were covered, but the obsessive focus on “AI” certainly didn’t help, as demonstrated by the bizarre reporting linked in the OP.
What are you referring to about Windows Hello?
Windows Hello existed long before AI became a trendy thing.
The instructions supported by NPU processors are designed for tasks like image recognition.
So, theoretically Hello might be quite a bit faster or more reliable (for a given time or energy budget) with an NPU.
The marketing is 100% drivel though.
Nothing to do with NPUs, as far as I know, it’s down to the dedicated video stream acceleration that ARM chips have carried for a while.
In practical terms, Windows Hello unlocks noticeably faster, just like a phone with facial recognition does. It also disproportionately handles video playback more efficiently, given the relative overall performance. Nothing new, either, it’s all stuff Apple got out of their own ARM transition, but way underrated in how much of a moment-to-moment difference it makes, particularly for stuff like the Surface Pro and other hybrids, where you want to handle it on battery and use it more like a tablet sometimes.
I’m still hoping that the somewhat irrational anger towards “AI” stuff subsides
I think this anger is linked to the irrational exuberance for “AI”.
Personally, I kinda hate AI. Not because of any sort of fear of job loss or anything like that. It’s because “AI” has been rolled out heavily in the Cybersecurity space, making my work life hell because of it. Models are only as good as their training and this means that any AI model which is going to spot anomalies in a network needs to spend a good amount of time being trained. However, what the vendors sell are touted as unsupervised models. They just need to spend a while on your network and they can automagically learn what “normal” is and then alert you on “abnormal”. This ignores the fact that you still need your analysts chasing false positives constantly from this black box. And that “black box” aspect is a major problem. You’ll get an AI/ML based alert with exactly fuck all in detail on why the alert triggered. If you’re lucky, you might get a couple log entries along with the alert, but nothing saying why those entries are suspicious.
I will grant that, there are a few cases where the “AI” in a product has worked. Mostly, it’s been in language processing. Heck, having an AI half-write a function for you in a tool you don’t use very often is quite nice. You almost always need to rework the results a bit, but it can get you started. But, my first question for any vendor talking about “AI Detections” is “how do we tune false positives?”. It’s just too big of a headache. And most of them try to downplay the need or dodge the question. Or, you have to babysit the model, effectively making it a supervised model. Which, fine. Just stop telling me how much time it’s going to save me, when I’m going to spend more time supervising the model than searching for threats in my environment. And, for fucks sake, design that shit to explain itself.
As for putting AI in my system. I can see a use case for language processing. Heck, I’d love to have the Star Trek style, “hello computer…” type stuff actually work worth a damn. Google and Siri are pretty close, though even those can be shit on toast when trying to do anything slightly complex. And having all that done locally, without having to send data “to the cloud” sounds great for privacy and security (until MS adds a keylogger as part of the OS). But, given how much time my GPU sits at or very near idle, I do wonder if the extra chip is worth the silicon or space.
In the end, I’m expecting this to go much the way TPM has. We’ll all end up with it in our systems, whether or not we know, care or use it. All because manufacturers just start soldering it on to everything. Maybe someone will find a good use for it eventually, distributed AI porn, maybe? But, like a lot of AI, it seems like a solution in search of a problem.
I’ve been extremely annoyed with all partisanship on this issue from the go. The absolutely insane range of promises, hyperbole and circlejerking among the AI techbros is absolutely insufferable. Meanwhile, the other side has been entirely content with getting a chance to repeat the dogpile they (deservedly) gave cryptobros and entirely unwilling to acknowledge any sort of nuance, often to the point of deliberate misinformation.
The only reason I’m not angrier at this is that, honestly, this whole thing is way less momentous than either side has claimed and I have bigger things to get mad about.
What the hell do I need a laptop with an NPU for? I can write my own emails and documents. I don’t give a crap about the paint and notepad AI additions. I seriously cant think of any use case for copilot plus.
And it’s hard to tell what the difference is. Apples ‘built from the ground up for AI’ chips just have more RAM. What’s the difference with CPUs? Do they just have more onboard graphics processing that can also be used for matrix multiplication?
The stupid difference is supposed to be that they have some tensor math accelerators like the ones that have been on GPUs for three generations now. Except they’re small and slow and can barely run anything locally, so if you care about “AI” you’re probably using a dedicated GPU instead of a “NPU”.
And because local AI features have been largely useless, so far there is no software that will, say, take advantage of NPU processing for stuff like image upscaling while using the GPU tensor calculations for in-game raytracing or whatever. You’re not even offloading any workload to the NPU when you’re using your GPU, regardless of what you’re using it for.
For Apple stuff where it’s all integrated it’s probably closer to what you describe, just using the integrated GPU acceleration. I think there are some specific optimizations for the kind of tensor math used in AI as opposed to graphics, but it’s mostly the same thing.
The idea is having tensor acceleration built into SoCs for portable devices so they can run models locally on laptops, tablets and phones.
Because, you know, server-side ML model calculations are expensive, so offloading compute to the client makes them cheaper.
But this gen can’t really run anything useful locally so far, as far as I can tell. Most of the demos during the ramp-up to these were thoroughly underwhelming and nowhere near what you get from server-side services.
Of course they could have just called the “NPU” a new GPU feature and make it work closer to how this is run on dedicated GPUs, but I suppose somebody thought that branding this as a separate device was more marketable.
Basically yes. They come with an NPU (Neural processing unit) which is hardware acceleration for matrix multiplications. It cannot do graphics. Slap whatever NPU into the chip, boom: AI laptop!
Matrix multiplication is also largely what graphics cards do, I wonder how the npus are different.
Modern graphics cards pack a lot of functionality. Shading units, Ray tracing, video encoding/deciding. NPU is just the part needed to accelerat Neural nets.
The Apple chips also have a wide interface to the RAM. That means you can run chatbots (LLMs) and other AI workloads that are memory-bound at crazy speeds compared to an Intel (or AMD) computer.
Really? How fast is the memory bus compared to x86? And did they just double the bus bandwidth by doubling the memory?
I’m dubious because they only now went to 16gb ram as base, which has been standard on x86 for almost a decade.
Depending on the chip, they have somewhere from 100 to 400 GB/s. I’m not sure on the numbers on Intel processors. I think the consumer processors have about 50 - 80 GB/s. (~Alder Lake, dual channel DDR5) Mine seems to have way less. And a recent GPU will be somewhere in the range of 400 to 1000 GB/s. But consumer graphics cards stop at 24GB of VRAM and these flagship models are super expensive. Even compared to Apple products.
The people from the llama.cpp project did some measurements and I believe the Apple “Metal” framework seems to outperform the x86 computers by an order of magnitude or so. I’m not sure, it’s been some time since i skimmed the discussions on their Github page.
Apple is also much faster because the integrated graphics are actually usable for LLMs.
The base M is just a big faster than an Intel/AMD laptop if you can get their graphics working. The M Pro is 2x is fast (as its memory bus is 2x as wide). The M Max is 4x as fast.
AMD is coming out with something more competitive in 2025 though, Strix Halo.
It’s a double whammy. No one wants AI. No one likes Windows 11.
