Apple, to me, is the winner in the end of this AI chapter.
Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.
I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
I think the question to me is whether Apple can ship affordable consumer devices with enough memory to run sufficiently useful models sufficiently fast.
Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.
There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]
Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.
With how expensive memory is, IDK if they can pull this off but I hope they do
Even if it's 2030 - that's a mere 3 years from now - it will take at least that long for the greater economy to figure out how to use LLMs. And probably a memory boom/bust cycle in that time too.
If Apple, in 2030 has a compute product that a worker can BYO(X) to work that has 2TB RAM, with an encrypted personalized context that can float between agents, then they will be in a very interesting place.
Cloud compute has been completely commoditized. Yes, local models will get better and better, but I think for the vast majority, it will continue to be both cheaper and more productive to use cloud based models, far into the future, where bulk purchasing, oversubscription, and discounted industrial electric rates can be taken advantage of.
Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break over a decade in the future, for comparable cloud models. Yes, there's privacy, etc, but now you're down into fractions of a percent of users who care about the government reading the memory of their computer.
Man, if all that makes Apple a "winner" then I'm dying to hear your assessment of Nvidia.
Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.
I don't think that's materially different from what you can do with Nvidia and AMD GPUs, besides the fact that you're limited to Thunderbolt interconnect.
"Harbor is currently in a private preview under Apple’s Claris brand"
There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1997) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.
It was pretty fast to get it set up. It could be better for sure, as it's not a first class concept the same way that it is in linux, but an hour of google search ai mode will get you where you need to go.
OMG after 25 years, Claris gets something beyond FileMaker (among the most legacy software still alive)? Claris would not have the engineering talent, so why would Apple use them as a vector?
Hopefully it won't be another preinstalled bloatware app like Image Playground. macOS enshittification has been greatly accelerating, even giving Win 11 Home a run for its money.
Apple, to me, is the winner in the end of this AI chapter.
Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.
I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
I think the question to me is whether Apple can ship affordable consumer devices with enough memory to run sufficiently useful models sufficiently fast.
Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.
There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]
Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.
With how expensive memory is, IDK if they can pull this off but I hope they do
[1] https://www.tomshardware.com/tech-industry/semiconductors/ap...
Even if it's 2030 - that's a mere 3 years from now - it will take at least that long for the greater economy to figure out how to use LLMs. And probably a memory boom/bust cycle in that time too.
If Apple, in 2030 has a compute product that a worker can BYO(X) to work that has 2TB RAM, with an encrypted personalized context that can float between agents, then they will be in a very interesting place.
Most likely Apple has been spending whatever they can to get in front of the memory market.
They definitely are, judging by their price increases.
Even so, DDR5 is so high-demand that it's likely that Apple can't outbid customers like Nvidia or OpenAI when they drive the margins they do.
Cloud compute has been completely commoditized. Yes, local models will get better and better, but I think for the vast majority, it will continue to be both cheaper and more productive to use cloud based models, far into the future, where bulk purchasing, oversubscription, and discounted industrial electric rates can be taken advantage of.
Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break over a decade in the future, for comparable cloud models. Yes, there's privacy, etc, but now you're down into fractions of a percent of users who care about the government reading the memory of their computer.
And if you're on a mobile device, any halfway powerful local model is going to cut your battery life way down.
Are you an apple user or investor.
If you are an investor, you should remember they are a fashion product more than a tech company.
If you are a user, you should remember they are a fashion company more than a tech company.
Man, if all that makes Apple a "winner" then I'm dying to hear your assessment of Nvidia.
Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.
The contrast I'm drawing is running these models locally vs on the cloud and how that would be a selling point well received.
I don't think that's materially different from what you can do with Nvidia and AMD GPUs, besides the fact that you're limited to Thunderbolt interconnect.
"Harbor is currently in a private preview under Apple’s Claris brand"
There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1997) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.
But you’re still in for a ton of pain if you want to run them headless right?
They desperately need to fix that.
It was pretty fast to get it set up. It could be better for sure, as it's not a first class concept the same way that it is in linux, but an hour of google search ai mode will get you where you need to go.
How so?
I love that they are bringing Claris back for this. Moof!
I'm just learning that the Claris brand still exists.
Only took them 3 years for something that's inevitably worse than lm studio.
Users will praise it still. What's the phrase, Apple distortion field?
OMG after 25 years, Claris gets something beyond FileMaker (among the most legacy software still alive)? Claris would not have the engineering talent, so why would Apple use them as a vector?
Hopefully it won't be another preinstalled bloatware app like Image Playground. macOS enshittification has been greatly accelerating, even giving Win 11 Home a run for its money.