Does the small feature dimension affect the sublimation of knowledge?
Small feature dimension has influence on knowledge extraction. According to relevant public information, Knowledge Sublimation is committed to compressing a heavy network with good performance but high consumption into a lightweight network. Using the output of the middle layer of the teacher model feature extractor as a hint, combined with knowledge discovery, a deeper and narrower student model is extracted.