• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Wang, Zihan (Wang, Zihan.) [1] | Xie, Junwei (Xie, Junwei.) [2] | Yao, Zhiping (Yao, Zhiping.) [3] | Kuang, Xu (Kuang, Xu.) [4] | Gao, Qinquan (Gao, Qinquan.) [5] | Tong, Tong (Tong, Tong.) [6]

Indexed by:

EI

Abstract:

The aim of Knowledge Distillation (KD) is to train lightweight student models through extra supervision from large teacher models. Most previous KD methods transfer feature information from teacher models to student models via connections of feature maps at the same layers. This paper proposes a novel multi-level knowledge distillation method, referred to as Normalized Feature Fusion Knowledge distillation(NFFKD). The proposed model learns different levels of knowledge to improve the network performance. We proposed to use the hierarchical mixed loss(HML) module to minimize the gap between the intermediate feature layers of the teacher and the student, and the teacher-student gap is reduced by normalizing the logits. Experimental results have demonstrated that the proposed NFFKD shows superiority over several state-of-The-Art KD methods on public datasets under different settings. © 2022 IEEE.

Keyword:

Computer vision Convolutional neural networks Deep learning Distillation Knowledge management Students Transfer learning

Community:

  • [ 1 ] [Wang, Zihan]College of Physics and Information Engineering, Fuzhou University Fujian Key Lab of Medical Instrumentation and Pharmaceutical Technology, Fuzhou University, Fuzhou, China
  • [ 2 ] [Xie, Junwei]Imperial Vision Technology, Fuzhou, China
  • [ 3 ] [Yao, Zhiping]College of Physics and Information Engineering, Fuzhou University Fujian Key Lab of Medical Instrumentation and Pharmaceutical Technology, Fuzhou University, Fuzhou, China
  • [ 4 ] [Kuang, Xu]College of Physics and Information Engineering, Fuzhou University Fujian Key Lab of Medical Instrumentation and Pharmaceutical Technology, Fuzhou University, Fuzhou, China
  • [ 5 ] [Gao, Qinquan]Fujian Key Lab of Medical Instrumentation and Pharmaceutical Technology, Fuzhou University Imperial Vision Technology, Fuzhou, China
  • [ 6 ] [Tong, Tong]Fujian Key Lab of Medical Instrumentation and Pharmaceutical Technology, Fuzhou University Imperial Vision Technology, Fuzhou, China

Reprint 's Address:

Email:

Show more details

Related Keywords:

Related Article:

Source :

Year: 2022

Page: 111-116

Language: English

Cited Count:

WoS CC Cited Count: 0

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Affiliated Colleges:

Online/Total:70/10035528
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1