NewsNTech
Model distillation is the mechanism behind the current dispute between US AI companies and Chinese developers. The technique trains a smaller model on the outputs of a larger one.
The student model learns to replicate its teacher's decisions, capturing a significant portion of the larger model's capability without access to the underlying training data or the compute investment behind it.
That decoupling is the concern the labs are pressing: a party that can observe a frontier model's outputs has a path to a capable model without reproducing the original cost.
At Y Combinator's annual Demo Day, CEO Garry Tan told founders the right response is to "do nothing." Frontier AI companies have accused China of using distillation to copy their models, framing it as a threat to competitive positions built on scale and proprietary training data.
Keep reading