🔍 Read the full analysis: Meet Inkling 2 Mini: AI Performance For Your Local Setup on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters and activating about 3 billion per token. It supports several inference runtimes and comes with a research variant that can use fewer passes through repeated layers. The company reports competitive benchmark results, but independent validation, practical hardware needs and real-world performance remain unclear.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for local AI setups. The company describes it as having about 26 billion total parameters, with about 3 billion active per token; it also offers inference support across several runtimes and a research variant with adjustable passes through repeated layers. The release reports strong benchmark results, but independent validation and hardware requirements are not established in the supplied material.
Inkling 2 Mini accepts text and image input and is described as having a 1 million-token context window. Its architecture uses 24 physical decoder layers. The first six and last six run once, while the 12 middle layers are reused in three passes, yielding 48 logical layer visits per token. The company says weight reuse in the middle section raises capacity without requiring a separate set of weights for every logical layer.
The model routes each layer visit to eight experts from a pool of 128, according to the release. Thinking Machines says the fixed-pass model is available through Transformers, llama.cpp and MLX, with day-one SGLang support for datacenter deployments. A separate research artifact, TML-d24-varloop-research, allows users to change the number of passes; the company says fewer passes can improve speed, potentially at a cost to quality.
The release also describes expert weights in NVFP4 and MXFP8 mixed-precision formats and a DSpark drafter intended to accelerate generation with a small memory cost. The supplied information does not give a complete model file size, minimum hardware configuration or comparable generation-speed measurements across consumer devices.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
What Local Model Users Gain
Inkling 2 Mini adds another option for users seeking a multimodal model they can run through local inference software. Its reported size, expert routing and reused layers are design choices aimed at combining model capacity with lower active computation and weight storage than a dense model of similar total parameter count might require. They do not, on their own, show how much memory the model needs or how quickly it will run on a particular computer.
For people comparing local models, practical value will depend on hardware compatibility, image performance, output quality and long-context behavior, not just the parameter count or context-window claim. Support in several inference tools may make testing more accessible, but actual results can vary with quantization, runtime, device and settings. The release makes a technical option available; it does not yet settle whether the model is a good fit for a given local setup.
The adjustable-pass research checkpoint may also interest developers studying the trade-off between inference speed and answer quality. Whether fewer passes deliver a useful speed improvement without an unacceptable quality change will require measurements across tasks and configurations.
Popular right now

Dyson Cyclone V10 Origin Cordless Vacuum

De'Longhi Magnifica Evo Fully Automatic Espresso Machine with Milk Frother, Silver

Vitamix VX1 Professional-Grade Blender, 64 oz, Black
As an affiliate, we earn on qualifying purchases.
How Its Repeated Layers Work
The central design choice is to run the same 12 middle physical layers three times, rather than assign separate weights to all 48 logical layer visits. The release says each pass begins with the same input from the first six layers and that learned messages carry information between passes. A learned gate controls how much information is carried forward. The technical description also says the model still executes all 48 visits and retains separate key-value and short-convolution state for each visit, so weight reuse does not mean all computation or memory costs disappear.
Thinking Machines compares Mini with its Inkling 2 Small model: both execute 48 layer visits per token, while Mini has a hidden width of 2,304 versus 4,608 and routes among 128 experts per layer rather than 384. The supplied material calls Mini substantially smaller but does not provide a complete file size or minimum hardware specification. It also notes earlier exploration of looping architectures, including Nanbeige.
The release’s benchmark claims compare Inkling 2 Mini with models including Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. Thinking Machines says Mini leads in 32 of 48 benchmarks and describes its standing on other tests, including FrontierCode v2.0, Terminal-Bench 4.0 and AutomationBench-AA v1.0.6. Those figures and characterizations are company-reported in the source material, which does not include a full score table or independent replication.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Performance and Hardware Still Open
The supplied material does not establish the company’s benchmark results through independent testing. It gives selected comparisons but no complete score table or reproducible third-party results. The claim that the model leads in 32 of 48 benchmarks is therefore a company-reported result, not an independently confirmed finding in the material available here.
Practical details are also missing: the source does not specify minimum memory requirements, full model file size or measured generation speeds on consumer hardware. Independent assessments of image handling and million-token context performance are not provided. Nor does the source fully assess the quality of the research checkpoint when run with one or two passes. A model card may contain further details, but it was not included in the supplied material.
Model Card and User Tests
The next evidence to watch for is the full model card, complete benchmark scores and reproducible outside evaluations. Tests using clearly identified hardware and runtime versions could clarify how much memory Inkling 2 Mini requires, how fast it generates text, and whether the reported image and long-context capabilities hold up in practical use.
Developers and local-model users can also compare the fixed three-pass model with the one- and two-pass research configurations, measuring speed and output quality across the same tasks. Until those results and configuration details are available, the model’s local performance and the value of its adjustable-pass option remain open questions.
Key Questions
What is Inkling 2 Mini?
Inkling 2 Mini is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it accepts text and image input and has about 26 billion total parameters, with about 3 billion active per token.
Can Inkling 2 Mini run locally?
The release lists support through Transformers, llama.cpp and MLX, which can be used for inference in different environments. However, the supplied material does not identify minimum hardware or memory requirements, so it does not confirm which consumer devices can run the model effectively.
What does the adjustable-pass version change?
The TML-d24-varloop-research checkpoint allows the number of passes through the repeated middle layers to be changed. Thinking Machines says fewer passes may speed inference, with a possible trade-off in quality; comparative measurements are not supplied here.
Are the benchmark results independently confirmed?
Not in the provided source material. The reported results are claims from Thinking Machines Lab; the material does not include a full score table or independent replication.
Primary source: Hugging Face · via ThorstenMeyerAI.com
