The Surprising Effectiveness of Deleting Weights in LLM Post-Training
What happened
Researchers found that zeroing fewer than 0.05% of a pretrained large language model’s weights, using a binary mask learned over the gate projection of transformer blocks, matches the performance of full fine‑tuning across reinforcement learning with verifiable rewards, supervised fine‑tuning, and on‑policy distillation. This method, called Deletion Tuning, produces a mask that is hundreds to thousands of times smaller than a full fine‑tuning checkpoint and causes far less forgetting of prior tasks.
Why it matters
The approach offers a much more storage‑efficient way to adapt LLMs and helps preserve earlier capabilities, addressing a key challenge in continual learning.
Is this a big deal?
Get stories like this every morning
The day's AI news in plain English, in your inbox at 8:30 IST. Free.
More AI news
Every Way to Get 192GB for Local AI (Without a $9,500 Mac) [video]Industry news · 14 Nov 2026DeepSeek V4 Pro 0813DeepSeek · 14 Nov 2026DeepSeek V4 Pro 0813DeepSeek · 14 Nov 2026Show HN: Portal2Bench – Can AI agents solve Portal 2?Industry news · 14 Nov 2026Anthropic Upgrades Claude Code Projects to Run Parallel AI Workers Hands-FreeAnthropic · 14 Nov 2026This page was collected from arxiv.org and summarised automatically by AIPulsee's pipeline. See how it runs.