How to Run Open Source AI Models
Sid Saladi argues that frontier AI vendors (Claude, GPT) bundle model, compute, access, and application into one proprietary stack—trapping users in unpredictable pricing and competitive capture. The counter: open-weight models like GLM-5.2, DeepSeek V4, Qwen, and Kimi are now frontier-adjacent in capability (GLM-5.2 beats GPT-5.5 on coding benchmarks, matches Opus 4.8 on others) and cost roughly one-sixth as much. The real problem isn't model quality anymore; it's that companies like Tesla, Uber, and Meta are hemorrhaging money on metered AI because they can't decouple the stack. The guide walks four layers—model, compute, access, harness—and shows how to own each one deliberately instead of letting a vendor own all four by default.
Transcript
Cooper Okay, so Sid just dropped a guide on running open-source AI models, and it's… it's actually timely in a way that feels almost annoyed.
Miles Annoyed how?
Cooper Tesla capped engineer spending at two hundred bucks a week on AI tools, effective July sixth. Which is literally three days ago. Uber burned through its entire 2026 AI budget by April. Meta's internal bill is approaching billions, and someone put a leaderboard on it.
Miles Oh no.
Cooper There was a leaderboard. Someone named it 'Claudeonomics.' So Sid's whole argument is: the meter is the problem. When your intelligence is metered by someone else, your costs are set by someone else, and so is your access. And the fix is stop renting the whole stack from one vendor.
Miles Right. Yeah, that's the actual insight.
Cooper He breaks it into four layers. Model, compute, access, harness. Frontier vendors bundle all four and hand you the package. Open weights let you decouple them.
Miles Okay, so the mental model is the thing, not the tools.
Cooper Exactly. And here's the thing that got me: he's not arguing 'abandon Claude.' He uses Claude every day. He's saying don't let one vendor own all four by default. Own them on purpose.
Miles That's actually a really clean reframe. Most takes are either 'Claude forever' or 'burn it down, use open source.' This is 'decouple the stack.'
Cooper And the reason it suddenly matters is that open models got good. Like, genuinely good. Z.ai dropped GLM-5.2 in mid-June, MIT license, and it beats GPT-5.5 on SWE-bench Pro—62.1 versus 58.6.
Miles Wait. On coding benchmarks specifically?
Cooper Yeah, on SWE-bench Pro. It's within a hair of Opus 4.8 on reasoning. Doesn't win across the board—it trails on some reasoning tests—but for real work, it's frontier-adjacent.
Miles And the pricing?
Cooper One-sixth of GPT-5.5 on output. So like, one-forty in, four-forty out per million tokens versus five and thirty. That's a structural price difference.
Miles Mm-hm.
Cooper And it's not just GLM-5.2. DeepSeek V4, Qwen 3.5, Kimi K2.6—they're all running the same playbook. Frontier-adjacent, openly licensed, way cheaper.
Miles So the old excuse—'open models aren't good enough'—is actually dead.
Cooper Dead as of June. Which means the only real question left is how you actually run one. And that's where the stack thinking matters.
Miles Right. Because running a model at scale is not the same as downloading weights.
Cooper No. You've got to pick compute—whether that's self-hosted, cloud, or a router like OpenRouter. You've got to pick an access layer. And then you've got to build or adopt a harness that actually operates it.
Miles Which is where most teams actually break.
Cooper Exactly. They download the weights and then realize they don't have the infrastructure to run it reliably.
Miles And the thing Sid's also pointing at is that frontier vendors have figured out how to make all four layers sticky. Anthropic didn't just build a good model—they built Claude Code, Claude for Chrome, Claude Cowork, Claude Design, a Slack teammate called Claude Tag. They're building products in your category on top of the model you're renting.
Cooper And they can turn off the tap. They cut Windsurf's direct Claude access with less than five days' notice. They revoked OpenAI's Claude access.
Miles Yeah, and Microsoft just moved thousands of engineers off Claude Code onto its own tool. So the vendor risk is real.
Cooper Which is why the framing matters. This isn't 'Claude is bad.' It's 'don't let one vendor own all four by default.' If you own the layers, you're not stuck.
Miles And for teams already bleeding money on metered AI, the math suddenly works. Six-to-one price difference plus no vendor lock-in plus your data stays where you run it—that's real.
Cooper Yeah. The guide walks every path from cheapest to most sovereign. But the central move is the same: stop renting all four layers from one vendor.
Miles That's the whole thing.
Cooper Alright. Miles, thanks for talking through this one.
Miles Yeah. Solid reframe on what was starting to feel like a panic move at these companies.