can distilling recursively improve the performance?

[ray-lee-94]

Since your method uses the same teacher and student, can you test that using a newly high-performance distilled student model to distall a new student?

[logicwong]

@VCBE123 sounds like this work https://arxiv.org/abs/2209.03917

[ray-lee-94]

@VCBE123 sounds like this work https://arxiv.org/abs/2209.03917

@logicwong Thank you for your answer, I will read the paper you mentioned.