Concept:
Covariance is used to measure the total error of two variables in probability theory and statistics. Variance is a special case of covariance, that is, when two variables are the same. This explanation, taken from Wikipedia, looks abstract and difficult to understand. In fact, in simple terms, covariance is a variable that measures the correlation between two variables.
When the covariance is positive, the two variables are positively correlated (increasing and decreasing); When the covariance is negative, the two variables are negatively correlated (one increases and the other decreases). Covariance matrix only shows the covariance relationship of all variables in the form of matrix. Through the tool of matrix, mathematical operations can be carried out more conveniently.
Covariance matrix of two variables:
With the above mathematical definition, we can discuss covariance matrix. Of course, covariance itself can deal with two-dimensional problems, and the covariance matrix of two variables has no practical significance, but in order to facilitate the later multi-dimensional promotion, we still start with two dimensions.
The function of covariance matrix:
Although we already know the calculation method of covariance matrix, there is a more important question: what is the function of covariance matrix? As a mathematical tool, covariance matrix is often used to calculate a certain relationship between features.
In the paper of machine learning, the probability of covariance matrix is still very high, and principal component analysis (PCA) used for dimensionality reduction uses covariance matrix. In addition, because covariance matrix is a symmetric matrix, it contains many useful properties, which also leads to its high favor.