Coursera's Ng machine learning, UFLDL have all seen it. If I remember correctly, the formula was given directly in Ng's machine learning. Although you may know how to solve it, it is not a problem even if you don't know how to finish your homework. Just follow the formula. I didn't see it clearly when I looked at it anyway. I think it's good to know the deep learning course UFLDL-Ufldl. If there are exercises, you will really have a deeper understanding of deep learning, but it is still not very clear. Later, I read Li Feifei's Stanford University CS 23 1N: Convective Neural Network for Visual Recognition, and I felt that my understanding of CNN had been greatly improved. Calm down and deduce the formula, think more and understand that reverse propagation is essentially a chain rule (although I knew it before, I was still confused at that time). All gradients are actually derived from the final loss, that is, the derivative of scalar to matrix or vector. Of course, I also learned a lot about cnn. It is also suggested that you not only complete the exercises, but also write a cnn article yourself. This process may make you learn a lot of more detailed and neglected things. Such a network can use the middle layer to build a multi-layer abstraction, just as we do in Boolean circuits. For example, when we are doing visual pattern recognition, neurons in the first layer may learn to recognize edges, and neurons in the second layer may learn to recognize more complex shapes, such as triangles or rectangles, on the basis of edges. The third layer will be able to recognize more complex shapes. And so on. These multi-level abstractions seem to give deep network the ability to learn and solve complex pattern recognition problems. Then, as can be seen from the example of line, some theoretical research results tell us that deep net is stronger than shallow net in essence.