Stanford researchers have developed a new language model architecture, Next-Concept Modeling (NCM), that could make AI substantially faster on existing hardware.
Most large language models generate text one token at a time. Each new token requires another pass through the model, repeatedly reading its weights from GPU memory. Moving this data can limit how quickly text is generated.
Diffusion language models generate multiple tokens in parallel, but still require repeated passes through the model to refine the whole passage.
NCM takes a different approach. It starts with a compact representation of the entire passage, capturing its broad structure. It then progressively adds detail, generating many parts simultaneously at each stage.
This allows NCM to generate text in fewer sequential steps than the diffusion models tested, potentially reducing repeated access to GPU memory and making better use of the processor's computing power.
This matters because memory bandwidth is often a major constraint on AI performance. NCM could reduce that constraint without requiring more powerful chips.
The researchers compared NCM with diffusion language models of similar size.
- Text generation was 10–31 times faster, with comparable or better quality on the benchmarks tested.
- Training required processing 4–5 times fewer tokens to achieve competitive results.
There are important qualifications.
The models tested were relatively small, and the speed comparisons were against diffusion models, not conventional large language models.
The potential hardware benefits have not been directly measured, and it remains to be seen whether the results hold at much larger scales.
Nevertheless, NCM offers a promising alternative to existing approaches, with the possibility of significantly faster AI generation on existing hardware.