Tag
#distillation
Reports tagged distillation.
Reports
2 reportsAug 2026
08-01 Deep Dive Distillation Mostly Reweights — Until the Teacher Adds Something New A weaker teacher can beat a stronger one when its token distribution overlaps the student's. In one 2026 study that overlap tracked successful distillation — but did not guarantee it. Jul 2026
07-31 Deep Dive Distillation Copies the Transcript, Not the Teacher A student model learns from a transcript, not from a mind — which is why capability transfers cheaply, why refusal quietly does not, and why the policy fight is aimed at the wrong part of the machine.