Abstract: Traditional knowledge distillation methods typically use soft target probability distributions to transfer knowledge from the teacher to the student model. However, this can lead to ...