Current location - Education and Training Encyclopedia - Graduation thesis - Does the small feature dimension affect the sublimation of knowledge?
Does the small feature dimension affect the sublimation of knowledge?
Small feature dimension has influence on knowledge extraction. According to relevant public information, Knowledge Sublimation is committed to compressing a heavy network with good performance but high consumption into a lightweight network. Using the output of the middle layer of the teacher model feature extractor as a hint, combined with knowledge discovery, a deeper and narrower student model is extracted.