Any intelligent system with powerful functions, light hardware and low cost is the first principle that developers should consider. It is the same as the "maximum output under production constraints" and "maximum utility under personal budget" revealed by economics. As Professor Shen Bin of Tongji University said, foreign countries focus on selling software, while domestic enterprises focus on selling products. In terms of software, some enterprises in Hangzhou have done better.
For machine vision and computer vision, complex systems such as automatic driving and artificial intelligence require higher efficiency, and convolutional neural networks are widely used in the field of intelligent vision. Recently, FaceBook team found a new way and put forward a new neural network compression method-Bitgoesdown, which greatly promoted the reduction of system hardware requirements. What enlightenment will this bring?
Small but refined
The FaceBook team compressed the image detection mask R-CNN (He) by 25 times and trained it with 8 V100 GPUs. The results show that the compressed model mask AP only drops by about 4%. The team also compared the compression results of image classification residual network models such as ResNet- 18 and ResNet-50, and found that ResNet-50 has the best performance when compressed to 5MB!
By the way, Mr. He just won the Young Scholar Award at the 3 1 Computer Vision and Pattern Recognition Conference last year. Previously, the student, as the first author, also won the best thesis awards of CVPR 2009, CVPR 20 16, ICCV 20 17 (Mar Award) and ICCV 20 17. In 2009, he became the first China scholar to win the "Best Paper Award" from CVPR, one of the three international conferences in the field of computer vision. Ming Kai is also the champion of Guangdong College Entrance Examination who walks Tsinghua. He graduated from the University of Hong Kong with a Ph.D. and entered FaceBook on 20 16.
He Mingkai's mask R-CNN is an extension of fast R-CNN, and a branch for predicting the target mask is added on the basis of fast R-CNN, which is parallel to the existing bounding box classification branch. Mask R-CNN is simple to train, and only increases or decreases the time consumption on the basis of faster R-CNN, and the running speed reaches 5fps. Moreover, Mask R-CNN can be easily applied to other tasks.
Open up a unique new road for yourself ―― develop a unique new style.
The compression of convolutional neural networks has been explored in academic circles. At present, the mainstream thinking mainly focuses on MobileNets, with high accuracy, but it is still far from the excellent state. This time, the FaceBook team will return to the traditional convolutional network framework, and the biggest feature is to pay attention to the activation number, not the weight itself. The specific implementation includes layer quantization and network quantization, and the specific implementation method is as follows:
In fact, this way of learning is unsupervised. The team used distillation technology to teach "student" network compression, which was proposed by Hinton et al. (Turing Prize winner).
References:
1.QbitAI "The new compression algorithm of Facebook benefits embedded devices";
2.ICCV20 17 Dr. He Mingkai's best paper mask R-CNN tutorial report;
3. Professor Shen Bin, Director of Mechanical Engineering Department of Tongji University, delivered a speech.
end
Visual system design is free.