Can Qwen3.8-Flash-Next Run Privately In 128 GB RAM? The 360 GB Catch
Qwen3.8-Flash-Next reportedly fits in 128 GB through an 87 GiB quant. See the 360 GB official-weight catch and how to verify local privacy.
Qwen3.8-Flash-Next can run locally on a 128 GB unified-memory computer through aggressive community quantization, but Qwen does not publish 128 GB as an official requirement. The official checkpoint is about 360 GB, the official FP8 version is about 186 GB, and one current community build compresses the model to an 87 GiB GGUF that reportedly occupies about 91 GB while running.
Watch The 30-Second Summary
That result is impressive, but it comes with catches. The test used an unmerged llama.cpp change and a manual crash fix. The machine was a high-end Ryzen AI Max+ 395 with 128 GB of unified memory. Its short-context speed does not prove that every 128 GB computer will load the same build, preserve model quality, support the full context window, or remain stable under long multimodal and agent workloads.
There is also a privacy catch. Downloadable weights make private local inference possible; they do not make every launcher, browser interface, agent tool, search connector, log, or cloud fallback private. You have to verify the complete data path.
What Is Confirmed
Qwen released Qwen3.8-Flash-Next on August 26, 2026 as an experimental preview of architectural ideas intended for Qwen4. The official model card describes a multimodal mixture-of-experts model with:
- 125 billion main-model parameters;
- 6 billion main-model parameters activated per token;
- an additional 51 billion n-gram embedding parameters;
- an additional 4 billion multi-token-prediction parameters; and
- a native context length of 262,144 tokens, with documented extension to one million tokens.
The release uses a hybrid architecture built around Gated DeltaNet and Qwen Sparse Attention. Qwen says the sparse-attention system selects micro-blocks rather than individual tokens, with the goal of reducing long-context latency. The model also accepts text, images, and video.
The official Hugging Face repositories show the scale more clearly than the “6B active” headline:
| Distribution | Approximate weight size | What the number means |
|---|---|---|
| Official full checkpoint | 360 GB | Downloaded model artifacts before runtime overhead |
| Official FP8 checkpoint | 186 GB | Lower-precision official weights before runtime overhead |
| Community UD-IQ4_XS GGUF | 87 GiB | Unofficial quantized file used in one 128 GB demonstration |
| Reported resident memory in that demonstration | About 91 GB | One machine, runtime, context, and build—not a universal requirement |
Qwen documents Transformers, vLLM, SGLang, and TokenSpeed support. Its model card recommends dedicated serving engines such as vLLM, SGLang, or KTransformers for production and high-throughput use. The default model behavior includes a visible thinking block, and the card documents controls for disabling thinking or deciding whether prior thinking blocks remain in the conversation context.
The official repository uses the Qwen Community 1.0 license, not Apache 2.0. Anyone planning commercial distribution or a hosted service should read the actual license text instead of assuming that every downloadable Qwen model has the same terms.
The release also has real attention. During this review, the official model was first on Hugging Face's trending list with roughly 3,700 likes. A same-day LocalLLaMA release megathread drew hundreds of votes and substantial hardware discussion.
What Is Still Unclear
Whether 128 GB Will Become A Supported Local Target
Qwen does not list 128 GB as a supported minimum. The strongest evidence is a community Strix Halo test using a 128 GB Ryzen AI Max+ 395. The tester reported an 87 GiB UD-IQ4_XS quant, about 91 GB resident memory at a 131,072-token configuration, roughly 23 tokens per second of generation, and about 390 tokens per second of prompt processing in the measured run.
Those numbers are useful evidence that the model can be made to fit. They are not a vendor benchmark, repeatability study, or guarantee for a conventional CPU-plus-GPU computer. The setup relied on an unmerged llama.cpp pull request and a manually described crash fix. Until support lands and is tested broadly, this is an expert experiment rather than a routine installation path.
How Much Quality The Community Quant Preserves
The 87 GiB build is far smaller than the 360 GB official checkpoint. Quantization can preserve impressive capability, but it is lossy. No independent evidence reviewed here establishes that this specific quant matches the official checkpoint across coding, vision, long context, tool use, multilingual tasks, or difficult reasoning.
The model loading successfully is only the first test. A useful evaluation should compare the exact quant against the official model on the tasks that matter to the operator, including failure cases and sensitive decisions.
What Long Context Costs On A 128 GB Machine
The reported run used a 131,072-token configuration, not proof of reliable use at the model's full 262,144-token native window or the optional one-million-token extension. Context cache, temporary buffers, vision inputs, concurrency, and runtime choices consume memory beyond the weight file.
Qwen's model card says one-million-token operation requires RoPE scaling and specific serving-engine configuration. That is a capability path, not evidence that an 87 GiB quant plus a million-token working set fits comfortably in 128 GB.
Whether The Preview Will Remain The Recommended Version
Qwen calls this release experimental and says Qwen3.8-Flash is the production-oriented official version with additional features such as default one-million-token context and built-in tools. Flash-Next is valuable for evaluating the new architecture, but production teams should not assume that its APIs, runtime support, license, quantizations, or recommended deployment stack are settled.
Why “6B Active” Does Not Mean A 6B Model In RAM
Mixture-of-experts models route each token through only part of the network. That reduces computation per token. It does not normally let a standard runtime discard every expert that is inactive for the current token, because a different token may need a different expert a moment later.
Qwen's “125B with 6B activated” line describes compute routing inside the main model. It does not erase the 125 billion main parameters, the 51 billion n-gram embedding parameters, or the 4 billion MTP parameters from the checkpoint.
This distinction explains an otherwise confusing result:
- only about 6 billion main parameters may participate in one token's routed computation;
- far more weights must remain available for future tokens; and
- the official artifacts still occupy hundreds of gigabytes.
Active parameters can help generation speed. Resident weights determine whether the model fits. Treating those as the same number is the central hardware mistake in many sparse-model headlines.
Can Qwen3.8-Flash-Next Really Run In 128 GB RAM?
Yes, one current community configuration says it can. The more precise answer is: a heavily quantized build can fit on one 128 GB unified-memory system with experimental software, substantial remaining caveats, and less than 40 GB of headroom before other workloads.
Unified memory matters. On the tested AMD system, the CPU and integrated GPU can use one large memory pool. A desktop with 128 GB of system RAM and a separate 24 GB GPU does not behave the same way. Some runtimes can divide work between GPU memory and system RAM, but transferring layers or cache across that boundary can reduce throughput sharply.
The practical hardware tiers look like this:
- Less than 96 GB available: The reported 87 GiB file plus runtime overhead is unlikely to fit without a smaller and potentially more destructive quant, storage offload, or other compromises.
- 128 GB unified memory: Demonstrated experimentally, with enough room for the reported 91 GB resident set but not unlimited context, concurrency, or desktop workloads.
- 192–256 GB unified or accelerator memory: More headroom for larger quants, context cache, vision, and runtime buffers, though official engine support still matters.
- Multiple data-center GPUs: The more conventional target for the official full or FP8 checkpoints and production serving.
Do not buy hardware from the file size alone. Check available memory after the operating system starts, the exact quant, supported runtime version, context target, image and video workload, expected concurrency, and measured tokens per second.
The LOCAL Audit For Private Inference
“It runs locally” answers where the model math executes. It does not answer where the surrounding system sends or stores data. Use this five-part LOCAL audit before placing sensitive material into any self-hosted model.
L — Load Path
Confirm that inference uses the downloaded checkpoint and a loopback or private-network endpoint. Some desktop apps advertise local models while retaining a hosted-model fallback. Disable fallback and verify what happens when the internet connection is unavailable.
Model download is a separate network event. Downloading weights from Hugging Face does not send later prompts to Hugging Face, but the downloader, package manager, or launcher may still check for updates.
O — Outbound Features
List every feature that can contact another service:
- web search;
- URL fetching;
- remote code sandboxes;
- speech recognition or synthesis;
- image and video tools;
- hosted embeddings;
- crash reporting;
- update checks; and
- agent tools or MCP servers.
A local model with cloud search is a hybrid system. That may be acceptable, but it should be an explicit decision. Network observation or an egress firewall is stronger evidence than a “local” badge in the interface.
C — Context And Copies
Find where prompts, outputs, uploaded files, extracted text, thinking blocks, embeddings, thumbnails, and caches are stored. Qwen3.8-Flash-Next preserves thinking blocks from historical messages by default unless the surrounding template changes that behavior. That can improve continuity while also increasing the sensitive context retained in a session.
Check application logs, terminal history, browser storage, temporary directories, backups, synchronization folders, and monitoring tools. Local data is still exposed to other users, malware, misconfigured shares, stolen devices, and unencrypted backups.
A — Access Boundary
Bind the inference server to the narrowest interface that works. A service intended for one computer should not silently listen on every network interface. If remote access is required, use authentication, encryption, firewall rules, and a dedicated account with limited permissions.
Also inspect the browser UI. A local web page can load remote scripts, fonts, analytics, images, or extensions. The model server and the interface are separate components with separate trust boundaries.
L — Lifecycle
Document the exact model hash, quantization, runtime commit, startup options, and configuration. Experimental support can change quickly, and the current llama.cpp path is not yet merged. Updates can alter networking, logging, context handling, or model compatibility.
Test deletion as well as installation. Know how to remove conversations, caches, derived embeddings, uploaded files, logs, and backups. “No cloud account” does not mean “no retained data.”
A Safer 128 GB Evaluation Plan
Start with synthetic data and a bounded goal. Do not make a new experimental runtime's first workload confidential source code, private medical material, legal documents, or customer records.
- Record the exact build. Save the model repository, quant filename and hash, runtime commit, unmerged patches, and startup arguments.
- Use a short context first. Confirm basic generation and memory use before increasing the context window.
- Measure the working set. Observe peak memory during prompt ingestion, generation, image input, and long conversations—not only after loading.
- Test without internet. Verify that ordinary chat still works and note which features fail or retry remotely.
- Observe egress. Check DNS and outbound connections during startup, chat, file use, tool calls, and shutdown.
- Inspect storage. Locate histories, thinking traces, caches, logs, and temporary uploads; test their deletion.
- Challenge the quant. Compare representative outputs with a higher-precision deployment when the decision is important.
- Keep tools off initially. Add search, code execution, file access, and agents one at a time, with separate permissions and logs.
The goal is not merely to make the model answer. It is to prove which data stays where under the exact workflow you plan to use.
How This Differs From The Other Qwen3.8 Releases
Qwen's recent naming makes size comparisons easy to flatten incorrectly.
The earlier Qwen3.8 flagship is a 2.4-trillion-parameter text model with roughly 95 billion parameters active per token. Its official BF16 checkpoint is about 4.89 TB and its reference production layouts are data-center scale.
The separate Qwen3.8-27B is a dense multimodal model whose official BF16 weights are about 55.56 GB. Community four-bit files near 17 GB make it plausible on ordinary enthusiast hardware.
Qwen3.8-Flash-Next sits between those stories in a non-obvious way. It is much larger than the 27B model in total weights, far smaller than the 2.4T flagship, and sparse enough to use about 6B main-model parameters per token. The official files still exceed a typical workstation, while the community 87 GiB quant makes a high-end 128 GB unified-memory system plausible.
That is why each release needs its own answer. “Qwen3.8 runs locally” is too broad to describe the hardware, quality, licensing, or privacy boundary of all three.
Where OpenVeil Fits—And Where It Does Not
OpenVeil is a hosted privacy-focused AI workspace for adults who want chat, private search, files, voice, images, video, and custom personas without maintaining a large local inference stack. In normal private-chat use, chat history is kept in the user's browser instead of as a normal server-side chat-history record. OpenVeil also says prompts, uploaded files, media, selected local history context, and AI outputs are not used to train foundation models.
That can be the simpler choice when your real requirement is private chat-history handling and useful hosted tools—not absolute local execution. There is no 87 GiB model download, experimental llama.cpp patch, memory tuning, inference server, or local security maintenance to operate.
The boundary matters. OpenVeil is not fully offline, anonymous, zero-log, or a local Qwen host. Active requests still require processing by OpenVeil and necessary providers. OpenVeil does not make a Qwen deployment private, inspect its network traffic, secure a local server, prevent prompt injection, or protect a compromised device.
If you need the strongest possible control over active inference and can operate the infrastructure correctly, a verified local model can provide a boundary that a hosted service cannot. If you want a privacy-focused hosted workspace without running a 100-plus-billion-parameter model, OpenVeil addresses a different and often more practical need.
Frequently Asked Questions
Is Qwen3.8-Flash-Next A 6B Model?
No. Qwen reports 125 billion main-model parameters with about 6 billion activated per token, plus 51 billion n-gram embedding parameters and 4 billion MTP parameters. “6B active” describes routed computation, not the total checkpoint resident in memory.
Will It Run On Every Computer With 128 GB RAM?
No. One community report demonstrates an 87 GiB quant on a specific 128 GB unified-memory AMD system using experimental llama.cpp support. Conventional split RAM and VRAM, different runtimes, operating-system use, longer context, and other applications can change whether it fits or performs acceptably.
Is The 87 GiB Quant Official?
No. Qwen's official distributions are about 360 GB for the full checkpoint and 186 GB for FP8. The 87 GiB UD-IQ4_XS file is a community quantization and may not match the official checkpoints on every task.
Does Running It Locally Make It Private?
It can keep inference local if the checkpoint, runtime, interface, tools, logging, storage, and network configuration are all controlled accordingly. A local model connected to cloud search, hosted speech, remote tools, telemetry, or an exposed server is not a completely local data path.
Does Qwen3.8-Flash-Next Support Images And Video?
Yes. The official model card describes native multimodal support for text, images, and video. Vision inputs also add memory and processing demands that a text-only load test may not reveal.
Can It Use A One-Million-Token Context On 128 GB?
The model card documents extension from 262,144 native tokens to one million with RoPE scaling and compatible serving engines. It does not establish that an 87 GiB quant plus the required working memory can use that window reliably on a 128 GB machine.
Should A Business Deploy The Flash-Next Preview?
Treat it as an experimental evaluation target. Review the Qwen Community 1.0 license, runtime maturity, quantization quality, security configuration, support path, and the production-oriented Qwen3.8-Flash alternative before relying on it.
The Bottom Line
Qwen3.8-Flash-Next has crossed an interesting threshold: a 125B sparse multimodal model with another 51B embedding parameters can reportedly run from an 87 GiB quant on one 128 GB unified-memory workstation. That is real community evidence and a remarkable engineering result.
It is not an official 128 GB requirement, a promise of full-context stability, proof that the quant preserves official-model quality, or automatic privacy. The official checkpoint is about 360 GB, the FP8 version about 186 GB, and current llama.cpp support is still unmerged.
If you test it locally, audit the entire system—not just the model file. Verify the load path, outbound features, stored context, access boundary, and lifecycle. The model can be local. Privacy still has to be demonstrated.