Open-Weight Models: Llama, Mistral, Qwen, DeepSeek

If you were paying attention during the three years from LLaMA 1 in early 2023 to Llama 4 in 2026, something that used to feel obvious stopped being obvious. In 2023 the frontier was a closed API and the open-weight tier was clearly behind. By 2026 that gap is a slice-by-slice story. On code, on math, on Chinese, on plenty of fine-tuned narrow domains, the best open-weight model is either at the frontier or within arm's reach of it. On complex multi-step reasoning and novel-domain instruction following it still trails, sometimes by a lot. Which half you live in depends entirely on what you are trying to build.

This post is the map of that landscape in 2026. Four families, what each one is known for, what "open" actually means once you read the license, and when self-hosting starts making economic sense versus paying a provider. I am going to try to stay out of the benchmarking weeds and stay in the part where the mental model lives.

The problem open-weight models solve

Frontier APIs are wonderful and they also come with a list of things you cannot do. You cannot run them offline. You cannot fine-tune most of them on your internal data without going through a hosted SFT pipeline on a curated subset of hyperparameters. You cannot audit their training mixture. You cannot guarantee they will still exist, unchanged, in eighteen months. You cannot take them onto a submarine. These are not hypothetical concerns. They come up in regulated industries, air-gapped deployments, research, adversarial red-teaming, and any application where the per-token bill at scale stops looking like rounding error.

Open-weight models, by which I mean models where the provider has published the parameter tensors under a license that lets you download and run them, are the escape hatch. You get the same artifact a production inference engine uses, you can fine-tune it with LoRA or full SFT without asking anyone, you can run it on your own GPU, and the bill becomes whatever your GPU cost is.

By 2026 this is a real and crowded space. Four families do most of the work.

Prereqs

This one sits pretty late. The things you should already have handles for:

• Pretraining and the shape of a transformer block, from the attention-from-scratch and FFN-as-memory posts.
• MoE layers from the dense-sparse-hybrid post, because two of the four families we cover today lean on Mixture-of-Experts heavily.
• LoRA / PEFT from lora-and-peft, because the thing you usually do with an open-weight model is fine-tune it cheaply.
• Inference engines from the arc-6 posts, because "run it yourself" means pointing vLLM or llama.cpp at a checkpoint.

Nothing in this post is new mechanism. It's the ecosystem view of mechanisms you already understand.

The openness axes

Here's the mental model I wish someone had handed me when I started looking at license pages. "Open" is not one axis. It is at least five, and they are independent.

Once you separate these, a lot of online arguments stop making sense. A Llama 3 checkpoint has weights you can download, commercial rights with a caveat, and fine-tune permission, but its training data is a black box and the license is not OSI-approved (the discrimination-based-on-use clause alone is a disqualifier). A Mixtral 8x7B checkpoint from the open tier is Apache 2.0, genuinely OSI-approved with no MAU caps and no use restrictions, but the training data is also a black box and Mistral's larger commercial models are closed. Qwen moved from a custom license to Apache 2.0 somewhere around Qwen2, and now the recent releases are genuinely permissive. DeepSeek has the most permissive weights license of the four and the most transparent training-report, and its license still isn't OSI-approved because of the acceptable-use clause.

The OSI nuance matters if you care about downstream distribution. The Open Source Initiative published its Open Source AI Definition in 2024 and deliberately set the bar high: the definition requires the training data description, the training code, and a license that allows unrestricted use, study, modification, and redistribution. Almost no weights-available model meets it. The field needed a word for "you can download and run it" that did not overclaim "open source."

The handle I carry around: the openness axes. When somebody says "is X open?" my first reflex is to ask which axis. Five questions, five answers, one at a time.

A minimal formalism: what a license actually governs

There is no math to reach for here, so instead, a tight schema of the terms that actually do work in these licenses. When you read any of them, these are the four knobs that tell you what you can and can't do:

Redistribute is whether you can put the weights on your own server for other people to download, and under what conditions (attribution, notice, no-sublicensing). Commercial is whether you can use the model or its outputs in a product you make money from, and whether there are scale caps (Meta's 700M MAU clause is the canonical example). Derivative is whether you can fine-tune, quantize, merge, distill, and redistribute the result, and whether the derivative inherits the license. Use restrictions are the acceptable-use policies: no weapons, no CSAM, no defamation, no election interference, whatever the provider decided to list.

Every license-page argument is an argument about one of these four slots. Once you see the slot, the argument usually simplifies.

How one of these models actually runs

From the inference engine's perspective, Llama and Mixtral and Qwen and DeepSeek all look almost identical. They are all decoder-only transformers. They all speak the Hugging Face transformers checkpoint format or can be converted to GGUF for llama.cpp. They all ship with a tokenizer (BPE, though the specific merges and special-token sets differ) and a chat template that says how to format a conversation into token IDs.

The three moving parts at runtime:

• Weights. A directory of model-*.safetensors shards plus a config.json telling the engine the hyperparameters (num layers, hidden dim, num heads, rotary base, MoE config if any).
• Tokenizer. Either a tokenizer.json (HF tokenizers format) or a SentencePiece file plus a vocabulary. Determines exactly which bytes produce which token IDs.
• Chat template. Usually a Jinja template in the tokenizer config that serializes [{role, content}, ...] into a prompt string. Each family has its own tags: Llama uses <|begin_of_text|> and <|start_header_id|>, Mistral uses [INST] and [/INST], Qwen uses <|im_start|> and <|im_end|>, DeepSeek has its own variant.

The architectural family differences show up in the shape of the transformer block itself. A Llama block is the vanilla dense recipe: attention with RoPE, then a SwiGLU feedforward that is roughly silu(xW_up) * (xW_gate) → W_down. Mixtral is the same skeleton but the feedforward is replaced by a router plus eight parallel experts, picking the top two per token. DeepSeek-V2 and V3 keep the MoE idea and add Multi-Head Latent Attention, which compresses the K and V tensors into a much smaller latent before caching, slashing KV cache memory roughly an order of magnitude. Mistral's classic 7B used sliding-window attention, where each token only attends to the last 4096 positions rather than the whole sequence, though the Mixtral 8x7B and 8x22B releases dropped sliding-window in favor of vanilla causal attention.

In practice, you point vLLM or llama.cpp at the checkpoint, it reads config.json, picks the right attention kernel and the right expert routing, and you get an OpenAI-compatible HTTP server. The ecosystem smoothed this out hard over the 2023–2026 window. "Download and serve a 70B model" went from a multi-day project to a two-line command.

Worked example

Here is the comparison everyone wants to see. One benchmark at a time, five models, on the same 0-100 scale.

Even in this tiny three-benchmark slice the shape is clear. On code, Qwen2.5 72B and DeepSeek-V3 are at or above frontier-adjacent on HumanEval. On general knowledge (MMLU), DeepSeek-V3 actually edges ahead of GPT-4's reported number. On MATH, the gap is larger and the frontier still leads, though Qwen and DeepSeek are both in the 60s. Llama 3 70B is a respectable generalist but does not win any of the three specific slices. Mixtral 8x22B shows the cost of being an older open-weight MoE: competent, but consistently behind the 2024-2025 releases.

I want to flag the phrase frontier-adjacent because I use it a lot. It means "matches frontier APIs on a specific slice while falling behind on others." That is the honest summary of the best open-weight models in 2026. Not "as good as GPT-4 or Claude." Not "toys you should ignore." Frontier-adjacent on the slices that happen to match what they were optimized for.

A few patterns to pull out. Qwen's multilingual strength, especially on Chinese and other CJK languages, is not subtle; the training mixture has the vocabulary and the data to back it up. DeepSeek's code and math lead is consistent across every benchmark I have looked at and comes from a combination of architectural efficiency (MoE buys more capacity per FLOP) and aggressive math/code data mixtures during pretraining and post-training. Llama's profile is "balanced competent generalist," which is exactly the Meta strategy: no single-slice win, but nothing embarrassing either, and the broadest downstream community.

Where open still trails closed: novel-task instruction following, multi-step reasoning that requires sustained planning, and safety behavior in domains the model wasn't specifically post-trained on. The frontier labs spend enormous amounts on these via RLHF and the post-training data mixture, and it shows.

Implementation sketch

Here is how little code it takes to load two of these models and compare their outputs.

# pip install transformers huggingface_hub accelerate torch
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
 
PROMPT = "Explain why matrix multiplication is associative, in two sentences."
 
# Any HF model id with the right license works here.
MODELS = [
    "meta-llama/Meta-Llama-3-8B-Instruct",
    "Qwen/Qwen2.5-7B-Instruct",
]
 
for model_id in MODELS:
    print(f"\n=== {model_id} ===")
    tok = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(
        model_id, torch_dtype=torch.bfloat16, device_map="auto"
    )
 
    # Each family has its own chat template; let the tokenizer handle it.
    messages = [{"role": "user", "content": PROMPT}]
    inputs = tok.apply_chat_template(
        messages, add_generation_prompt=True, return_tensors="pt"
    ).to(model.device)
 
    out = model.generate(inputs, max_new_tokens=180, do_sample=False)
    # Strip the prompt; print only the generated continuation.
    reply = tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True)
    print(reply)

Hugging Face's transformers gives you a uniform interface over wildly different architectures. AutoTokenizer finds the right BPE tables and chat template, AutoModelForCausalLM finds the right config and instantiates the right block (dense, MoE, whatever), and apply_chat_template handles each family's special tokens for you. The only thing that changes when you swap Llama for Qwen is the string in MODELS. For the big models (70B+), you'd add quantization flags (load_in_4bit=True via bitsandbytes) or reach for vLLM, but the shape is identical.

Llama models require accepting Meta's license on the Hugging Face page and authenticating with huggingface-cli login. Mistral, Qwen, and DeepSeek are ungated on recent releases and just download. This is itself a signal about the openness axes.

Economics: self-hosting vs API

This is where most "should we use open weights?" conversations actually live. The per-token price comparison is the natural starting point, but it buries the real variable.

The crossover sits at low QPS. Roughly: an on-demand H100 at around $2.50 per hour, serving a 70B-class model at peak throughput of maybe 150 output tokens per second, delivers about 0.54M output tokens per hour, which comes out near$4.60 per million tokens at full utilization. If the API is $3 per million, the API wins at every QPS below full utilization, because the self-host bill keeps running while the GPU is idle. Push utilization up and the self-host number drops fast, and at steady high QPS you can easily be under$1/Mtok.

The gotchas on the self-host side are not in the graph because they are not in per-token terms: engineering time to run the stack, auto-scaling for bursty workloads, evals on each quantization level, patching CVEs in your inference server, and the fact that you are now paying for idle GPUs overnight. For a lot of teams, the math says "use the API until you know you are actually saturating a GPU, then revisit." The lines on the chart are real, but the decision is almost always about what's around them.

The one place this math flips is data-sensitive workloads. If the reason you are self-hosting is that your data legally cannot leave your network, the break-even QPS is zero, because the API isn't an option. The other place is heavy fine-tuning: if you are burning a LoRA adapter per customer, the open-weight path gives you derivative rights the API path often does not. Both of these get framed as "economics" and neither is really about per-token dollars.

Misconceptions

"Llama is open source." This misconception motivated half of this post. Llama is open-weight. The weights are available under the Llama Community License, which permits commercial use with a >700M-MAU clause, restricts training competing LLMs on Llama outputs, and doesn't disclose the training data. It's not OSI-approved. The reason this distinction matters is not pedantic: it governs what you can redistribute, what you can build on top of, and what you owe back. Call it weights-available or open-weight and the rest of the sentence composes correctly. Call it "open source" and you're going to have a conversation with legal.

"Open-weight models are always behind the frontier." In aggregate, on a broad eval suite, yes, still. On specific slices in 2026, no. DeepSeek-V3 matches or beats GPT-4 on MMLU by its own reported numbers. Qwen2.5 leads on HumanEval. Llama 3.1 405B was competitive with the frontier for a full release cycle. The honest version of this statement is "the best open-weight model is behind the best frontier model on the hardest benchmarks, but is frontier-adjacent on many specific tasks, and is often ahead of older frontier models."

"Self-hosting saves money." Sometimes. At high QPS with good utilization, it saves real money. At low QPS with an idle GPU, it costs more, often dramatically. The natural framing is per-token, but the actual driver is utilization. The question to ask yourself is not "is the self-host price lower" but "am I going to saturate this GPU for enough of each day to pay for it."

"Mixtral 8x7B is a 56B-parameter model." By total parameter count, it's actually about 46.7B, not 56B. The name "8x7B" is misleading because only the feedforward layers are replicated across experts; the attention parameters are shared. By active parameters per token, it routes top-2 of 8 experts, so inference FLOPs are closer to a 12B-13B dense model. This is exactly the point of MoE. Memory bandwidth and VRAM still pay the ~47B tax because all experts live on the GPU, but the compute bill is the small-model one. This matters for choosing a host: you need the VRAM of the bigger model and the latency of the smaller one.

What's next

This post mapped the four open-weight families in 2026 along the five openness axes, walked through the architectural signatures of each (dense FFN, MoE, MLA, sliding-window), compared them on a benchmark slice, and traced the break-even math between self-hosting and APIs. The durable handle to carry forward: "open" is not one question, it is five independent ones, and the interesting open-weight models are frontier-adjacent on specific slices rather than uniformly behind.

The next post covers The Hugging Face ecosystem: how the Hub, transformers, tokenizers, datasets, safetensors, and peft became the connective tissue that turned a pile of checkpoints into a functioning ecosystem.

Additional reading (and watching)

• Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685. The paper that made fine-tuning open-weight models at laptop scale tractable.
• Bai, J., et al. (2023). Qwen Technical Report. arXiv:2309.16609. The original Qwen1 paper. Qwen2 (2024) and later shifted most open-weight releases to Apache 2.0, published in the Qwen2 and Qwen2.5 technical reports.
• Open Source Initiative. (2024). The Open Source AI Definition 1.0. The canonical standard against which "open source AI" is judged. Deliberately strict on training-data description and use-restriction clauses.
• DeepSeek-AI. (2024). DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. arXiv:2405.04434. Introduces Multi-Head Latent Attention (MLA) and details the training-data mixture and cost for V2.
• DeepSeek-AI. (2024). DeepSeek-V3 Technical Report. arXiv:2412.19437. Reports V3's MMLU, HumanEval, MATH, and related benchmark numbers alongside V2-to-V3 architectural changes.
• Meta. (2024). The Llama 3 Herd of Models. arXiv:2407.21783. The Llama 3 and 3.1 paper, including the 405B release. Source of Llama 3 70B benchmark numbers used in the comparison viz.
• Qwen Team. (2024). Qwen2.5 Technical Report. arXiv:2412.15115. Qwen2.5 72B scores on HumanEval, MMLU, and MATH, plus the Apache 2.0 license note.
• Jiang, A. Q., et al. (2024). Mixtral of Experts. arXiv:2401.04088. The Mixtral 8x7B / 8x22B paper, including the top-k routing scheme and MMLU / HumanEval numbers.
• Artificial Analysis. (2025–2026). LLM Performance Leaderboard. Provider-aggregated benchmark and pricing comparisons across open and closed models, updated continuously. Source for the rough 2026 consensus profile.
• Chiang, W.-L., et al. (2024). Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference. arXiv:2403.04132. The human-preference ELO leaderboard that underpins a lot of the "frontier-adjacent" language in this post.
• Jiang, A. Q., et al. (2023). Mistral 7B. arXiv:2310.06825. The original Mistral 7B paper, including the sliding-window attention design that Mixtral later dropped.
• Touvron, H., et al. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971. The first Llama paper, the opening shot of the modern open-weight era.
• Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155. The InstructGPT / RLHF paper that defines the post-training gap open-weight labs are still closing.
• Meta. (2024). Llama 3 Community License. The license text itself. Worth reading once to internalize what "weights-available but not open source" means concretely.
• Fedus, W., Zoph, B., & Shazeer, N. (2022). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv:2101.03961. The scaled-MoE paper whose routing ideas show up, modified, in Mixtral and DeepSeek.
• Kwon, W., et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv:2309.06180. The vLLM / PagedAttention paper. The serving substrate that most open-weight deployments sit on in 2026.