- one one hand, the 8GB RAM Mac mini with M6 as cheap as it maybe is an absolute waste of money when it comes to running LLMs
- on the other hand the most expensive machine is right here
- what is the cheapest most expensive machine you can buy capable of providing you the best ROI for running LLMs locally?
18 days ago [-]
justauserid 18 days ago [-]
Get a personal computer with 48-64 GB RAM (or maybe even 32 GB). Whenever your model usages go beyond that – there's those APIs/subscriptions just for that. You are golden for most (really most) of the work.
But if your use-cases need extremely heavy-duty local models then just open your bank vaults and get the one you need.
paxy 18 days ago [-]
Can't really answer that without knowing your use case. What is the largest model you need to run? What are you going to do with it? What tps do you need?
For just trying out local models and running occasional queries a 16GB mac will run an 8B-20B dense quantized model just fine.
If you are running at the scale of OpenAI then a data center full of blackwells will have the best ROI.
vivzkestrel 18 days ago [-]
the use case is getting the best bang for your back per dollar
take an 8 year life cycle for all machines and think of the cheapest expensive machine that you can buy which ll give you a decent llm to run
simonh 17 days ago [-]
Cheapest expensive? I'd say the M5 Pro Mini with 64GB RAM. That's certainly not cheap, but not ludicrous, and fairly capable. An M6 mini with 32GB would be the budget option and quite limited, you'd probably want to replace it after say 4 years.
To do significantly better than that with a Studio, say M5 Max with 128GB would cost twice as much again. It would probably be more than twice as performant and much more capable, but you'd be getting into very big bucks that IMHO would only be justifiable for paying pro work.
mjsweet 17 days ago [-]
I recently bought the M5 Pro Mini with 64 GB of RAM, and it feels excellent. It matches the Max in CPU performance while running Qwen 3.8 27b. Although it is slow, I can complete tasks at what I would call Opus 4.6–4.8 levels. What surprises me is using Claude Code on it. The speed of building with that harness and testing iOS apps is impressive… full end-to-end control of Xcode through to TestFlight. This is a major upgrade from my old M1 Pro MBP.
hebetude 18 days ago [-]
Buy one of the 128gb m4 max when people sell them to buy m5 ultras. It runs fine on the 27b models and qwen3.8 can actually run tools and perform tasks well. I use it for private agents, voice processing (no elevenlabs criminal pricing) and some photo manipulation though it's not very efficient at this task.
illusive4080 18 days ago [-]
Probably 64GB MacBook Pro and run a 27B model on it comfortably.
WarOnPrivacy 18 days ago [-]
When the page first loads, it shows a different price for ~1sec. It quickly overwrites that and displays the much larger, eye watering figure.
Loading the page with default parameters, probably from cache, then applying the specified parameters
ChrisRR 18 days ago [-]
The state of the web when every site that is just text and images, for some reason uses javascript
immutlex 18 days ago [-]
Has any individual/SMB purchased this yet? Aware the ROI isn't there for most individuals that don't heavily use LLM for coding or multi agentic workflows -- but for those that it does make sense this is ~$68k cheaper than the next alternative making this the most realistic option for consumer grade access to frontier models.
Specifically interested if anyone has ran GLM-5.3 (753B MoE) in 4-bit on a 512GB Mac Studio. Looking for real world usage numbers and/or any benchmarks you might have performed.
Thanks!
nsagent 18 days ago [-]
They haven't shipped yet (people who purchased the moment it became available have delivery dates near the end of the month). That said you can estimate it's speed by looking at the 256GB version for models with similar numbers of active parameters (GLM 5.3 has 40B active parameters). So for 4-bit you could compare to what people are getting Qwen3.8 27B at 8-bit (which is around 2000 tok/s prefill and 70-80tok/s token generation [1]). It might be a little higher or a little lower due to the different architecture and kernel optimizations.
GLM-5.3 would be bottlenecked by the M5's GPU, here. I don't have any hard numbers, but Apple Silicon is known for weak prefill performance on larger models (yes, even MoEs).
evanjrowley 18 days ago [-]
Good to know. Private GLM-5.3 is really the one use-case I can imagine for a system like this - It does very well with cybersecurity tasks, which are either blocked for most people by current providers, or on their way to getting blocked based on the regulatory/political trends. Such a capability could become very valuable when supply becomes artificially limited, although I expect the Chinese labs will be the last hold-outs. Imagine not being able to do security hardening on your own tech because government-imposed guardrails decide your legitimate prompts appear "risky". In such a world, what would an unconstrained cyber-capable model be worth?
cromka 18 days ago [-]
I think 2xGB10 might be a better option but don't quote me on that.
pmarreck 18 days ago [-]
is this a hardware or software limitation?
bigyabai 18 days ago [-]
Both. Apple Silicon could have the best of both worlds if it supported dGPU/eGPU tech, but Apple depreciated that about a decade ago. So you're stuck with fast RAM and a slow GPU.
pmarreck 14 days ago [-]
My understanding was that apple's integrated GPU was perhaps not equivalent to top-tier nvidia but was like a low to mid tier nvidia?
Tepix 18 days ago [-]
Crescent island will be cheaper and might ship earlier. It could also be faster.
drsalt 18 days ago [-]
apple always charged ridiculous numbers for ram and disk space.
jpitz 18 days ago [-]
What are you comparing this to, and for how much, and what sort of benchmark numbers is it getting? I'd love to run better cheaper hardware. I'll gladly pay you a reasonable finders fee.
jonplackett 18 days ago [-]
You can get it up to 29k with max storage.
DerekL 18 days ago [-]
By comparison, the Macintosh IIfx with 4 MB of memory and a 160 MB hard disk was about $28k in today's dollars.
So, effectively just as worth it for those who need it.
evanjrowley 18 days ago [-]
All roughly the same price, for a consumer who has won the lottery:
- 1x 512 GB Mac Studio with the 4 TB disk storage option.
- 4x Nvidia GB10s (total 512 GB RAM) each connected with a 100 GiB/s link to a Mikrotik CRS504-4XQ-IN.
- 3x Ryzen AI Max+ 495 192GB systems (total 576 GB RAM), connected via a 40 GiB/s link directly to each other.
mrheosuper 18 days ago [-]
What is the 40GiB link ?
SomeHacker44 18 days ago [-]
Probably USB4.
mrheosuper 18 days ago [-]
Well USB4/TB4 is 40 Gigabit/s, so 8 times slower than what they are expecting.
simonh 18 days ago [-]
Does anybody actually expect bandwidth to be expressed in bytes, rather than bits?
TheHalfDeafChef 17 days ago [-]
Your AI Max+ 495 would be bottlenecked by ~300gb/s no?
TheHalfDeafChef 17 days ago [-]
Your AI Max+ 495 would be hobbled by ~300gb/s bus no? So muuuuch slow inference
mannyv 18 days ago [-]
Ship date is 3/5/27 for the 512GB model.
Oddly enough, Apple Upgrade isn't an option.
tonyedgecombe 18 days ago [-]
5/3/27 if you are in the UK.
ChrisRR 18 days ago [-]
5/3/27 if you're in one of the ~150 countries that use date first
27/5/3 if you're in one of the ~15 countries that use year first
5/3/27 If you're in the 3 countries that use month first
hnfong 18 days ago [-]
Guys let's all be reasonable and write the date normally:
2003-05-27
Thanks.
simonh 17 days ago [-]
ISO is the way to go.
mannyv 17 days ago [-]
I usually write it 2026-Mar-03 in email these days because you never know which locale people are in.
frr149 18 days ago [-]
The elephant in the room: how much of those 512 Gb is usable by the GPU?
sithadmin 17 days ago [-]
75% by default, ~90% with manual overrides.
vr46 18 days ago [-]
Will one be enough, is my only question?
osti 18 days ago [-]
To be safe, just buy four and you'd be able to run the current biggest open weight models.
Naru41 18 days ago [-]
I hate to sound like Sam Altman, but is there actually any point in having 512 GB of RAM other than just pure tech hoarding?
Philpax 18 days ago [-]
Running inference on LLMs that can fit in 512GB. For example, you would be able to run either GLM-5.3-Flash or DeepSeek V4.1 Flash 1017 on this, both of which are very respectable models. It's up to you, of course, if ultimate privacy is worth $25k.
- one one hand, the 8GB RAM Mac mini with M6 as cheap as it maybe is an absolute waste of money when it comes to running LLMs
- on the other hand the most expensive machine is right here
- what is the cheapest most expensive machine you can buy capable of providing you the best ROI for running LLMs locally?
But if your use-cases need extremely heavy-duty local models then just open your bank vaults and get the one you need.
For just trying out local models and running occasional queries a 16GB mac will run an 8B-20B dense quantized model just fine.
If you are running at the scale of OpenAI then a data center full of blackwells will have the best ROI.
take an 8 year life cycle for all machines and think of the cheapest expensive machine that you can buy which ll give you a decent llm to run
To do significantly better than that with a Studio, say M5 Max with 128GB would cost twice as much again. It would probably be more than twice as performant and much more capable, but you'd be getting into very big bucks that IMHO would only be justifiable for paying pro work.
1st load ref: https://i.postimg.cc/YC5V5yZH/prerewrite.png
Specifically interested if anyone has ran GLM-5.3 (753B MoE) in 4-bit on a 512GB Mac Studio. Looking for real world usage numbers and/or any benchmarks you might have performed.
Thanks!
[1]: https://omlx.ai/benchmarks/performance?model=qwen3.8+27b&chi...
https://en.wikipedia.org/wiki/Macintosh_IIfx#Models
https://en.wikipedia.org/wiki/Apple_Lisa
Oddly enough, Apple Upgrade isn't an option.
27/5/3 if you're in one of the ~15 countries that use year first
5/3/27 If you're in the 3 countries that use month first
2003-05-27
Thanks.