1: \begin{abstract}

2: 

3: 

4: Transformer-based detectors~(DETRs) are becoming popular for their simple framework, but the large model size and heavy time consumption hinder their deployment in the real world.

5: While knowledge distillation(KD) can be an appealing technique to compress giant detectors into small ones for comparable detection performance and low inference cost.

6: Since DETRs formulate object detection as a set prediction problem, existing KD methods designed for classic convolution-based detectors may not be directly applicable.

7: In this paper, we propose \textbf{DETRDistill}, a novel knowledge distillation method dedicated to DETR-families.

8: Specifically, we first design a Hungarian-matching logits distillation to encourage the student model to have the exact predictions as that of teacher DETRs.

9: Next, we propose a target-aware feature distillation to help the student model learn from the object-centric features of the teacher model.

10: Finally, in order to improve the convergence rate of the student DETR, we introduce a query-prior assignment distillation to speed up the student model learning from well-trained queries and stable assignment of the teacher model.

11: Extensive experimental results on the COCO dataset validate the effectiveness of our approach. Notably, DETRDistill consistently improves various DETRs by more than 2.0 mAP, even surpassing their teacher models.

12: % The student model with our DETRDistill even outperforms its teacher model between three baselines.

13: 

14: % For example, DETRDistill consistently achieves over 2.0 AP performance gains on the COCO val set when Conditional DETR, Deformable DETR and AdaMixer served as teachers, respectively.

15: 

16: \end{abstract}

17: