I recently decided to try out local AI again on my 64GB MacBook Pro, running it against real projects instead of quick demos. So when a Hacker News thread argued that Qwen 27B is good enough, I wanted to check that against the size of task I actually ship, not a to-do app demo.

It isn’t. Go plug it into a large codebase like OpenEMR or Moodle, give it a task that actually needs to get done right, and it will fail you every time unless the task is a plain text change. Even then I’d be nervous.

The short version:

  1. Qwen 27B and the 35B A3B variant are impressive for quick questions, not for real agentic coding work.
  2. The industry doesn’t have a benchmark that captures what “good enough” means in practice.
  3. Local model sizes are growing faster than memory bandwidth, so the speed-versus-capability tradeoff keeps getting worse.
  4. The economics don’t favor local models either once you compare a $7,000 MacBook Pro against a subscription.

Impressive, but not agentic

I’ve run Qwen 27B. I’ve run the 35B A3B. They’re genuinely impressive for what they are. Will I keep them on my Mac? Sure, if the internet’s down or I’m on a flight and want a quick answer on how a library works, why not.

But claiming they have a real agentic experience is just outright wrong. They can’t automatically do much of anything beyond basic changes, closer to what Aider could pull off back in 2024.

They will often get lost in larger projects, and you have to spend more time up front making changes really tight plus being a watch dog. That is not my definition of agentic.

Nobody has a good benchmark for this yet

This reminds me of the overclocking community in the early 2010s: brand new territory, no rigorous way to measure results, so people fall back on feel. Someone runs their to-do app benchmark, the model does better, it “felt” like it did the task in one shot, and that becomes the evidence.

Similarly, many of my friends spent more time trying to get 3 more FPS than they did actually gaming. The fun for many local AI enthusiasts is in pushing the boundaries, not in actually producing something.

The industry’s own benchmarks don’t capture the nuance either. SWE-bench says a model hits 93%. That doesn’t mean it can accomplish 93% of programming challenges; it means it cleared 93% of a specific, narrow test set. The other number that matters and barely gets discussed is time horizon: how long can a model run unsupervised before it falls over. We’re getting better at measuring that, but it’s nowhere near rigorous enough to say a local model matches an Opus or a Sonnet.

I need my code today

Feed 200K tokens of context into one of these local models on an M5 with 128GB of memory and you could be waiting 23 seconds just for prompt processing. If it needs to generate 64,000 tokens back, the A3B variant is actually fast about it. The 27B parameter model is not: you’re looking at roughly 10 minutes to find out it screwed up five minutes in, or that your spec was wrong to begin with.

Programming has always been iterative. Stretch the gap between attempts and validation that long, and every project takes longer to ship while the outcome gets worse, because you never have unlimited time to write software.

The math on a $7,000 MacBook Pro

I’ve been a $200/month Claude Max subscriber for a full year. I’m still three years from breaking even on a MacBook Pro built to run local inference well, and I only own a 64GB MacBook Pro because my regular workload is diverse: Moodle development, mobile app work, a lot of stuff open at once. I didn’t buy it to run local inference. It’s good at it, but it’s not my main workhorse.

The counterargument I hear is that token costs will eventually catch up to you. That’s not really true anymore. Models like GLM 5.2 and DeepSeek V4 are a fraction of the cost of the frontier labs, and $1,000 covers months of usage unless you’re the kind of person who genuinely loves burning tokens.

Where local actually wins

I’m not anti-local AI. I think it’s critical for the right job. The clearest example: I built a project that locally transcribed a health startup’s entire records, faxed labs, plain text notes, everything, into a structured dataset, because the business owner was rightfully privacy conscious and wanted nothing touching the cloud.

I ran it for almost 13 hours straight for 33 patients. It burned out my travel Anker charger. I should have used the MacBook’s own charger instead of the 140-watt travel brick, but that brick gave up first. Local AI handled that job well, as long as you’re willing to wait.

Bigger models, same memory bandwidth

The local models everyone ran 2 years ago were all 7b and topped out at 13b. Now the workhorse size is 27 to 35 billion, and it keeps climbing because people demand more capability. What isn’t climbing at the same rate is memory bandwidth. The new M5 gained roughly 20% more bandwidth. The model sizes people actually rely on have nearly doubled in the same stretch.

So if you find a 35 billion parameter model you like and move up from 27 billion, congratulations, you just signed up for a roughly 30% slower model. People respond by splitting their workflow: a small model for basic tasks, a stronger one for advanced tasks. But the entire point of using an AI assistant is not having to make that call yourself. Let the model tell you what’s complex. Small local models struggle to do that.

They also struggle with coherency and recall across large context. On the medical document project, I found this out the hard way trying to fit an entire patient’s lab history into context at once. It just didn’t work, across roughly seven different models: Gemma variants, MLX builds, 4-bit, 8-bit, quantization-aware training. What finally worked was processing documents in small batches instead. Nobody really talks about that kind of nuance, and skipping it is exactly what sets people up to fail.

You’re only 2 years away from needing a new work horse.

Not your day job. Not yet.

You will not be running your day job on Qwen locally. Nobody wants their Mac hitting 110°C while a YouTube video stutters in the background. That’s not hyperbole either: when I run local inference at full tilt, my monitors flicker and disconnect. Probably a dock bug, but I doubt I’m the only one whose dock chokes on it.

Maybe in a couple of years you’ll offload to something like a Mac Studio running as a dedicated inference server, doing the workhorse work in the background. That would genuinely be great, since it would immediately undercut the monopoly the frontier labs currently have on serious AI work.

Until then, stop telling people a 27 billion parameter model is all they need for their day job. It isn’t.