At the base, the architecture wars are over
Most large organisations still buy AI by brand. Procurement asks which model, from which vendor, at which tier. Two years ago, that was the right question. In 2026, it is a smaller part of a larger one.
The base of the model has converged. Through 2024 and 2025, leading open releases increasingly settled on the same broad architecture: a mixture-of-experts transformer, a hybrid attention pattern that stacks a small fraction of full-attention layers on cheaper variants, rotary positional encoding, and multi-token prediction. Mamba, state-space models, and linear attention reached the frontier, then reappeared as components inside hybrid blocks rather than as replacements. The closed flagships are widely believed to follow the same broad recipe.
What feels new is happening below the brand. Decisions the vendor used to make in private are now public, named, and selectable. DeepSeek-V3 shipped in December 2024 as the first frontier-scale release with every layer named: 671 billion parameters with 37 billion active per token; native FP8 training; multi-head latent attention; multi-token prediction; and a fine-grained mixture-of-experts router. DiffusionGemma, released in June 2026, swapped the decoder on the same Gemma 4 backbone and shipped more than 1,000 tokens per second on a single H100. The procurement decision is now a stack of six choices. Most teams are still treating it as one.
A buyer of AI used to choose a brand. A buyer of AI now chooses a stack. The brand is one layer of six.
Why this matters now
Two things happened over the same 18 months. The vendors converged on the base, and the layers above the base opened up.
The convergence is the easier story. Three patterns became near-universal in open releases: rotary positional encoding, a mixture-of-experts router, and the hybrid attention pattern. Two more became defaults within the year: native FP8 training and multi-token prediction. By the time DeepSeek-V3 published its technical report in December 2024, every choice it described had been reproduced or refined elsewhere. The base of the model is a standard recipe now, and the debate has moved off it.
The opening-up is the harder story because it changes the buyer’s job. Decisions the vendor used to make – which tokeniser to use, which attention variant, what precision to train in, how to adapt the model to your data, whether to distil it down to something cheaper – are now choices a buyer can name. Each has a small set of known options. Each has reproducible numbers attached. And each has visible consequences for cost, latency, and quality on the workloads that matter to a buyer. The list of decisions grew by roughly a factor of six in 18 months, and most enterprise buyers have not yet noticed.
If your team is still answering ‘Claude or GPT or Gemini, and which tier’, you are answering the question that mattered in 2024. The question that matters in 2026 is which tokeniser will handle your domain text, which attention variant fits your typical context length, which precision your regulator will tolerate, and whether you want to fine-tune the frontier model or distil it into something you can own.
The transformer remains the base. The bill of materials around it has opened up, and the unit of decision has moved with it.
What sits below the brand
There are five named decisions below the brand. Each one used to be the vendor’s secret. Each one is now public.
The first is the tokeniser – the layer that decides how text becomes the numbers the model can read. The default, byte-pair encoding, is well understood but biased. It compresses common English well, fragments medical abbreviations and gene names, and handles long-tail languages less efficiently. Byte-latent transformers, published by Meta in December 2024, skip the tokeniser entirely and learn to chunk raw bytes by entropy as they go. For a biopharma team working with pharmacovigilance reports, where ‘NSAID’ and ‘IL-6 inhibitor’ are routine, the tokeniser is no longer a default to inherit. It is a procurement decision.
The second is the attention variant – the layer that decides how cheaply the model can hold long context. Standard attention scales with the square of the sequence length, one reason long prompts cost so much. Several alternatives now exist as drop-in choices. Grouped-query and multi-query attention reduce the cache by sharing heads. Multi-head latent attention, introduced by DeepSeek, compresses the cache into a low-rank latent and reconstructs it on demand, cutting key-value memory by 2.7 to 4.7 times relative to grouped-query peers. FlashAttention rewrites the matrix multiplication to be cache-aware. Sliding-window and sparse-attention variants trade fidelity for cost. The choice between them changes the economics of long-context workloads by close to an order of magnitude.
The third is training precision – the arithmetic the model runs in. Until 2024, the default was bfloat16. DeepSeek-V3 was the first frontier-scale release to train natively in FP8, with a fine-grained quantisation scheme that handles numerical outliers tile by tile. FP8 cuts training cost and memory in half relative to BF16. FP4 is now arriving in production: Hopper-class GPUs support FP8 natively; Blackwell extends Tensor Core support to FP4 and FP6. For a serving team measuring cost per token, precision is the single largest lever.
The fourth is adaptive computation – the layer that decides whether every token gets the same FLOPs. Mixture-of-Depths, published by Raposo and colleagues at DeepMind in April 2024, routes cheap tokens past expensive blocks entirely. It is the sibling of mixture-of-experts: where mixture-of-experts routes between experts, mixture-of-depths routes between depths. Same trick, different axis.
The fifth is adaptation and distillation – what becomes yours after you have bought the base. LoRA, QLoRA, and DoRA fine-tune a frontier-scale base on a single GPU with a small adapter you can own, version, and audit. Distillation copies the behaviour of a frontier teacher into a smaller student and has become a production pipeline. The DeepSeek-R1 distilled series, released in January 2025, shipped six dense students from 1.5 billion to 70 billion parameters; several of them beat GPT-4o and Claude 3.5 Sonnet on mathematics and code reasoning at a fraction of the inference cost.
Five layers. Each named. Each selectable. Each with measurable consequences for the workloads that actually run a business.
A senior practitioner can name a default and a leading alternative for every layer. A buyer who treats them as defaults is making the choices anyway.
Where this sits
This is a structural shift within a largely stable architecture. The transformer remains the transformer. What has changed is that the bill of materials around it has opened up.
The lineage is short. The 2017 paper laid the base. By 2022, the first layer-level choices – LoRA and FlashAttention – were standard infrastructure. December 2024 was the clearest marker: DeepSeek-V3 shipped with every layer publicly named and benchmarked. June 2026 made the decoder itself a selectable layer when DiffusionGemma replaced autoregressive generation with parallel diffusion on the same backbone. The closed flagships are widely believed to share these defaults. The frontier converged on the base and unbundled above it.
Lineage of layer-level decisions. The newest move sits at the decoder, the layer most buyers still treat as part of the brand.
What this means by sector
Biopharma
pharmacovigilance · clinical-trial submissions · regulator-facing text · sovereign data
The tokeniser stops being a default and becomes a procurement decision. Medical abbreviations and gene names fragment differently under different tokenisers, and the fragmentation shapes downstream extraction quality. Biopharma teams running into these limits are already training byte-level or byte-latent tokenisers on their own corpora. Structural shift.
Insurance
claims triage · underwriting copilots · fraud signals · line-of-business classification
Workloads come in volume, and the per-token economics dominate. Distilling a frontier teacher into a 7B or 14B student tuned on each line of business beats the API on cost and, on the workloads that matter, tends to match it on accuracy. Keeping the API for everything is expensive; the useful discipline is knowing which workloads to graduate. Economic shift.
Manufacturing
edge inference · vision-plus-text agents · sovereign workloads · factory-floor latency budgets
FP4 inference on factory hardware brings frontier reasoning inside the firm’s network for the first time. That shifts the deployment question from whether to deploy AI on-prem to which layer of the stack the on-prem deployment shares with the cloud-served version, and whether the two stay in sync. Operational shift.
Financial services
trader-floor latency · high-volume call paths · long-context analysis · per-token cost at scale
Speculative decoding and multi-token prediction collapse the latency budget on hot paths. Multi-head latent attention reshapes the cost curve on long-context analysis. A serving team that has not measured both is leaving headline performance on the table. Performance shift.
Three questions worth asking
01 Is our AI procurement framed as a model decision or a stack decision?
Which layers below the brand are we treating as defaults, and have we tested whether those defaults are right for the workload we run most?
02 On our highest-volume workload, what is the cost and accuracy gap between the frontier API and a distilled-then-tuned student on hardware we own?
A measurable question. An engineer can produce both numbers in a week: per-1,000-token serving cost on representative traffic, and accuracy delta on a held-out eval.
03 Of the five layers below the brand, which two would change our economics most if we owned them, and which is closest to feasible in the next 12 months?
Tokeniser, attention variant, precision, adaptive computation, adaptation-plus-distillation. The answer is rarely ‘all five’ and rarely ‘none’.
Together, the answers tell you whether the model is the unit of decision for your organisation, or whether the choice has already moved beneath it.
Where this lands
For most AI workloads, the frontier API will remain the right answer. Layer-level decisions earn their keep at scale, or where a sensitivity threshold – a regulator, a latency budget, a sovereignty mandate, a domain that drifts away from English – forces the question. This is not a universal recommendation.
The strongest evidence is public. DeepSeek-V3 showed that a 671 billion-parameter mixture-of-experts could be trained natively in FP8 for roughly 2.7 million H800 GPU-hours, with multi-head latent attention compressing the key-value cache to about 70 KB per token against 192 to 328 KB per token for grouped-query peers. The DeepSeek-R1 distilled series showed that a 32 billion-parameter student could reach 72.6% on AIME 2024 and 94.3% on MATH-500, outperforming non-reasoning frontier models several times its size. DiffusionGemma ships more than 1,000 tokens per second on a single H100 from a 26 billion-parameter backbone, at the cost of some headline benchmark quality. Three independent releases, three different layers of the stack, the same lesson: the layer below the brand is where the economics move.
Working out which layers matter for a specific organisation, and where the engineering investment earns its keep on the workloads that actually run the business, is what Brightbeam does. If your team is working through that question, we would welcome the conversation.
For the technical reader
Six choices that used to be the vendor’s, named with numbers attached.
The mechanism
The transformer absorbed its challengers. Mamba, state-space models, and linear attention each reached the frontier and then retreated, reappearing as components inside hybrid blocks. By 2026, the structural recipe is consistent: a mixture-of-experts transformer with hybrid attention – a small fraction of full-attention layers stacked on cheaper variants – FP8 or FP4 training, rotary positional encoding, and multi-token prediction. Below the brand, five layers are now publicly named in open-weights releases.
Tokenisation moves from byte-pair encoding through byte-level to byte-latent transformers that skip the tokeniser entirely. Attention efficiency moves from full attention through grouped-query and multi-query attention to multi-head latent attention and sparse-attention variants. Precision moves from BF16 through native FP8 – DeepSeek-V3, December 2024 – to FP4 with hardware co-design on Blackwell. Adaptive computation arrives as Mixture-of-Depths – Raposo et al., DeepMind – routing tokens past expensive blocks. Adaptation runs from LoRA through QLoRA to DoRA. Distillation has become a production pipeline: the DeepSeek-R1 distilled seriesshipped six dense students from 1.5B to 70B from a single frontier teacher.
The numbers
DeepSeek-V3: 671 billion total parameters, 37 billion active per token; native FP8 training across all major matrix multiplications with fine-grained per-tile and per-block quantisation; 2.664 million H800 GPU-hours for pre-training; multi-head latent attention reduces the key-value cache to about 70 KB per token against 192 to 328 KB per token in grouped-query peers – a 2.7 to 4.7× reduction. Numbers measured on H800 SXM with batch and context as reported.
DeepSeek-R1-Distill: the 32B Qwen-based student reaches 72.6% on AIME 2024 and 94.3% on MATH-500; the 70B Llama-based student reaches 70.0% and 94.5%. Both outperform GPT-4o and Claude 3.5 Sonnet on these benchmarks despite being a fraction of the size. The 1.5B and 7B students collapse on multi-step logic, dropping more than 50 points against the teacher on the harder tasks.
DiffusionGemma – Google DeepMind, June 2026 – is a 26B mixture-of-experts with 4B active per token; more than 1,000 tokens per second on a single H100 under Apache 2.0; 256K context; multimodal input. It trails standard Gemma 4 on every headline benchmark, a trade Google says is worth it for code infilling, in-line editing, and latency-bound agentic loops.
Where these numbers stop holding
Long-context tail above 32K flattens MLA’s advantage; mixture-of-experts routing overhead can swallow MTP and speculative-decoding gains at large batch; small distilled students – ≤7B – drop sharply on multi-step logic; FP8 numerical stability degrades on very long contexts unless the quantisation scheme is fine-grained.
What to do about it
INSTALL huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B for the distilled reasoning baseline; vllm>=0.7 for serving with MLA support; peft and bitsandbytes for QLoRA fine-tuning. Apache 2.0 for the Qwen-based distills; Meta community licences for the Llama-based variants.
TRY Take your highest-volume classification or extraction workload. Generate 200 to 500 supervised examples from the frontier API, QLoRA-fine-tune R1-Distill 7B or 14B on a single GPU in about four hours, then benchmark against the frontier API on your held-out eval. One afternoon to set up, overnight to train.
MEASURE Per-1,000-token serving cost on your representative traffic – not benchmark traffic. Accuracy gap to frontier on your eval – not theirs. P95 latency at your typical context length. Key-value cache memory at your typical batch. State the units. State the conditions.
WATCH Distillation collapses tail-case reasoning – 1.5B to 8B variants drop sharply on multi-step logic. FP8 numerical stability degrades on very long contexts unless the quantisation scheme is fine-grained. Mixture-of-experts routing overhead can erase MTP and speculative-decoding gains at large batch. QLoRA plus mixture-of-experts serving stacks still have rough edges. Licences vary by R1-Distill variant.
READ DeepSeek-V3 Technical Report for the canonical native-FP8-plus-MLA-plus-MTP recipe; Byte Latent Transformer for the tokeniser-free direction; Mixture-of-Depths for adaptive computation. One starting point per direction, not a reading list.
This Perspective began as a discussion inside Brightbeam’s Digital Intelligence: State of the Art and Beyond (DI SOTA and Beyond) programme, a standing forum where the people building production AI systems argue through what is real, what is hard, and what is merely fashionable. The thinking here is collective. The argument was sharpened in the room before it was written down.