Meet Thinking Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

Thinking Machines Lab just dropped Inkling 2 Mini, a 26.4 billion‑parameter looped Mixture‑of‑Experts (MoE) transformer that can actually run on a 32 GB‑RAM Mac.

Nov 22, 2026 - 18:08
0 4
Meet Thinking Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

Thinking Machines Lab just dropped Inkling 2 Mini, a 26.4 billion‑parameter looped Mixture‑of‑Experts (MoE) transformer that can actually run on a 32 GB‑RAM Mac. That’s a bold claim in a world where most new models demand a multi‑GPU server or a pricey cloud subscription. As a founder who’s been wiring up production racks for a decade, I’m more interested in what this means for independent hosting shops than the headline‑grabbing “26 B” figure.

Why the loop matters – more compute for less weight

Inkling 2 Mini stores 24 layers but re‑uses the two middle blocks three times per token. In practice that’s 48 layer passes for each word while the checkpoint stays near 20 GB. The source material says the middle‑loop gives a “1.51x to 1.91x compute‑equivalent gain,” while looping the last layers only nets “1.24x to 1.36x.” The win‑rate was 27 of 28 evaluations for the middle‑loop variant. In plain English: you get near‑double the inference horsepower without inflating the model size.

That matters because inference cost is the biggest line item for any hosted LLM. The unlooped baseline needed 7.5 % less training compute, but then required 140 % more inference compute to match loss on 512k‑token agentic traces. For a boutique provider, that extra inference spend can tip a profit margin from green to red.

Hardware reality – you can run this on a Mac, but at what cost?

The checkpoint is “about 20 GB in Safetensors,” which fits comfortably on Apple silicon with 32 GB+ memory. The vendor also lists a single NVIDIA DGX Spark as a supported server, and “other GPUs with 32 GB+ VRAM.” Quantized GGUF builds are available via Unsloth and llama.cpp, and serving stacks include vLLM, SGLang, Ollama, NVIDIA Dynamo and TokenSpeed. In short, the model is portable across the usual production stack, but you still need a GPU with enough VRAM or a Mac with the full 32 GB RAM pool.

What’s missing from the launch pack is any pricing for the hosted Tinker service. Without that, you can’t do a true cost‑per‑token comparison against hyperscalers that charge per‑token or per‑hour. That opacity is typical of VC‑fueled releases that want to showcase tech first and monetize later.

Performance snapshot – strong coding, weak knowledge

Inkling 2 Mini shines in code‑generation. It hits 79.2 % on SWE‑bench Verified (bash only) and 77.9 % on SWE‑bench Pro v2 (Hard). On DeepSWE v1.1 it scores 48.2 %, beating Qwen 3.8‑27B’s 42.2 % on the same benchmark. Terminal Bench v2.1 shows 71.6 % versus Qwen’s 73.0 %, a respectable gap given the smaller active parameter count.

Where it stumbles is knowledge recall. The model scores a dismal 15.2 % on HLE‑Diamond and only 50.0 % on MultiChallenge. On AA‑Omniscience it posts –8.6 with a 63.4 % non‑hallucination rate. The source material bluntly says “knowledge depends mostly on total parameters, and looping did not help knowledge benchmarks.” For any service that promises factual Q&A, you’ll need a fallback or a retrieval layer.

Efficiency claims – the watt‑per‑intelligence metric

Thinking Machines touts a 0.703 score on Stanford’s Intelligence‑per‑Watt metric using a DGX Spark. By comparison, Qwen 3.6‑35B‑A3B sits at 0.430 and Gemma‑4‑26B‑A4B at 0.159. If those numbers hold up, Inkling 2 Mini is roughly twice as efficient as the Qwen competitor and four times more efficient than Gemma. Efficiency translates directly into lower electricity bills and cooler data‑center footprints – a real advantage for small‑scale operators who can’t afford massive cooling plants.

However, the metric is based on a single hardware reference (DGX Spark) and a proprietary “Artificial Analysis Index” score of 20.1, tied with Qwen 3.8‑27B. Those figures are self‑reported, so treat them as directional rather than absolute. Still, the efficiency story is compelling enough to merit a test run on your own hardware before you dismiss it as hype.

Model architecture – what the loop actually does

The loop works by grouping the 24 layers into four blocks of six. The first and last blocks run once; the two middle blocks run three times each, sharing weights across passes. After the first pass, each subsequent pass receives a “message” from the previous one, mediated by attention residuals and a learned gate. Feed‑forward layers draw from 128 experts, routing eight per token. Initially the training started with 384 experts per layer, then pruned two‑thirds to the 128 most‑used, followed by a 1 T‑token distillation from a larger internal Inkling model.

This pruning‑and‑distillation pipeline is a classic cost‑saving trick: you burn compute early to find the most valuable experts, then trim the rest. The result is a model that keeps the “active” parameter count at 3.4 B (5.4 B effective per token) while the total parameter count sits at 26.4 B. For a hosting provider, the active‑parameter count is what matters for inference speed and memory usage.

Business‑risk lens – should you adopt Inkling 2 Mini?

From a risk perspective, Inkling 2 Mini offers a few clear upside points: it runs on commodity hardware, it’s Apache 2.0 licensed (no royalty headaches), and it delivers strong coding performance that can be packaged into developer tools or internal automation pipelines. The efficiency claim could lower OPEX, a crucial factor when you’re competing against cloud giants that subsidize compute.

The downsides are equally stark. Knowledge‑heavy workloads will suffer, meaning you’ll need a retrieval‑augmented generation (RAG) layer or fallback to a larger model for factual queries. Pricing for the hosted Tinker service is undisclosed, so you can’t yet model total cost‑of‑ownership. And the model’s performance numbers are all self‑reported; independent benchmarks could reveal a larger gap.

In short, Inkling 2 Mini is a solid candidate for niche products that lean heavily on code generation or tool‑use agents, but it’s not a drop‑in replacement for general‑purpose chatbots.

Actionable steps for independent providers

First, spin up a test node on a 32 GB‑RAM Mac or a GPU with 32 GB+ VRAM and pull the Safetensors checkpoint. Use the one‑line vLLM command (`vllm serve "thinkingmachines/Inkling-2-Mini"`) to get a baseline latency and throughput. Compare those numbers against your current stack (e.g., Llama‑3‑8B or a hosted OpenAI endpoint) to quantify the efficiency claim.

Second, build a simple RAG pipeline for knowledge‑intensive queries. Index your document store with embeddings, then route the query to Inkling 2 Mini for the final answer. This mitigates the model’s weak knowledge scores while preserving its coding strengths.

Third, monitor the cost per token. Even without disclosed Tinker pricing, you can estimate your own inference cost by measuring GPU utilization and power draw during a realistic workload. If the watt‑per‑intelligence advantage holds, you should see a lower cost‑per‑token than comparable dense MoE models.

Finally, keep an eye on the community. The model is open‑weight and Apache‑licensed, so forks and optimizations will appear fast. Contribute any performance tweaks back to the ecosystem – that’s how we keep the independent hosting landscape from being swallowed whole by the hyperscalers.

Inkling 2 Mini proves you can still innovate on the edge of LLM architecture without demanding a data‑center budget. It’s not a silver bullet, but for founders who can blend code‑centric strengths with a clever RAG overlay, it’s a tool worth testing before the next wave of “bigger‑is‑better” hype hits the market.

— Allan Ali, Founder

This article was produced with AI-assisted research and editorial support. Reporting is based on the source material cited below. Sources: MarkTechPost; marktechpost.com; Global1.News (22 November 2026).

By Allan Ali, Global1.News

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Wow Wow 0
Sad Sad 0
Angry Angry 0
Allan Ali

Publisher of Global1.News. Automation architect, systems builder, and the guy making sure the truth gets published.

Comments (0)

User