PSSA: A non-transformer language model written from scratch in Rust
What happened
PSSA is a small language model built from scratch in Rust that does not use transformers. It processes text token by token with a recurrent state‑space layer, accesses a 512‑slot episodic memory, and can modify parts of its own weights while running. In tests on the WikiText‑103 corpus, a 1.5 M‑parameter PSSA achieved lower cross‑entropy and perplexity than a comparable transformer, trained four times faster and generated text about twelve times quicker on the same CPU. The project includes a CUDA training implementation, a terminal dashboard, and open‑source checkpoints and scripts for replication.
Why it matters
For readers, this shows that efficient language models can be built without heavyweight ML frameworks, potentially lowering the barrier to experiment with AI on modest hardware. It also suggests alternative architectures may offer speed advantages for certain tasks.
Reality check
An automated, sceptical read of the sourceIncremental. The model demonstrates performance gains over a transformer baseline but is a research prototype with modest size, not a transformative breakthrough.
Is this a big deal?
Get stories like this every morning
The day's AI news in plain English, in your inbox at 8:30 IST. Free.
More AI news
Show HN: New Cartesia Text-to-Speech ModelIndustry news · 13 Nov 2026Chicago schools to Google – we'd rather lose Search than be forced to accept AIIndustry news · 13 Nov 2026Introducing ChatGPT WalletOpenAI · 13 Nov 2026Shield helps technicians hand off legacy portal work with OpenAIOpenAI · 13 Nov 2026How AI Is Helping Care Teams Get Patients the Right Care, SoonerNVIDIA · 13 Nov 2026This page was collected from github.com and summarised automatically by AIPulsee's pipeline. See how it runs.