Information theory was established by American mathematician C.E. Shannon, and Shannon's two papers "Mathematical Theory of Communication" and "Communication in Noise" published in Bell Journal of System Technology in 1948 and 1949 laid the foundation of modern information theory. Starting with the research of information measurement to be solved in communication, he established an information theory only used in communication systems, or Shannon information theory, also known as narrow information theory.

Shannon abstracted the form of communication system into the model shown in figure 1- 1 after deeply studying various complex communication systems. The model consists of six parts, and the significance of each part is:

Figure1-1Shannon's communication system model

1. Source and destination

Source is the source of information, which can form and send a message or a series of messages to be transmitted to the receiving end. There are many sources, not only people can send out information, but all objects in nature can send out information actively or passively and become sources. The information receiver is the information sink, which can receive information and copy it to achieve the purpose of communication. The receiver can be a person, such as a geologist, or a machine, such as various receiving instruments used in geophysical exploration.

2. Channel and noise

A channel is a medium for transmitting information and a channel for information exchange between a source and a destination. In geological exploration, information (such as electromagnetic field) sent by underground geological body (source) will pass through rocks in the crust and/or air on the ground and be transmitted to the destination, so the overlying strata and/or air of the source are channels.

When information is transmitted in the channel, it is often disturbed by noise. Noise encountered in communication systems can be divided into two categories: external noise and internal noise. Noise often affects the communication effect and causes some distortion. Therefore, the noise must be reduced as much as possible.

In geological exploration, noise is generally called interference.

3. Coding and decoding

A code is a list of symbols and some constraints that must be observed when arranging these symbols. Using these symbols and observing the corresponding constraints, information is transformed into signals, and this process is coding.

When the signal sequence is output through the channel, it must be decoded and copied into the message before it can be sent to the receiver. Decoding is just the opposite of coding, so decoding is the inverse transformation of coding.

The above parts constitute Shannon's communication model (see figure1-1).

Shannon model can be used not only in communication systems, but also in other non-communication fields. In a non-communication system, it can be called an information circulation system. Taking enterprise management as an example, information is provided by grass-roots units and other interested parties, and the leading organs understand and collect information through various channels, and then process and make decisions on the information. Here, the grass-roots units are information sources, the leading units are information sinks, and various channels for understanding and collecting information are channels. This is also a simple information flow system. From this point of view, geological prospecting can also be regarded as a complex information flow system (which will be described in chapter 3).

Second, the definition of information in narrow sense information theory

The development of any science has gone through two stages, namely qualitative research and quantitative research. Qualitative is the basis of quantitative, and quantitative is the accuracy of qualitative. Only when a science changes from qualitative research to quantitative research can it become an "accurate science", and information theory must be measured if it is to develop into an "accurate science".

In order to measure information, it is necessary to give an accurate definition of information. Shannon defined information as the content of uncertain things in the message (that is, things that were not known before). For example, if someone tells you something you already know, there is no information in the news. On the other hand, the less you know about the event before you are told, the more information you have in this information. For an event that must happen or must not happen under given conditions, when the conditions are met, someone tells you that this event has happened (for the event that must happen) or this event has not happened (for the event that must not happen). This news has no message for you.

If an event may or may not occur under given conditions, and it has a certain probability, when the conditions are met, someone tells you that the event has occurred or not, and this message contains information for you.

Three. Measurement of information [1]

According to the conventional method, when a quantity is expressed quantitatively, it is often compared with an appropriate standard. Obviously, this method is not suitable for the quantitative representation of information, so the measurement of information has become a long-term unsolved problem. Shannon, on the other hand, has made great contributions in this respect, making information quantitative.

According to Shannon's definition of information, we completely abandon the specific content of information, formalize information, and quantitatively describe information from the perspective of probability theory. The reasons are as follows: first, the sink only copies the signal sent by the source, regardless of its content; Secondly, before the signal sent by the source is received by the sink, the sink does not know when and what the signal sent by the source is, which is random. Therefore, the signal sent by the source can be regarded as a random event for the sink.

Let the probability of random event A be P(A), with 0≤P(A)≤ 1. If P(A)= 1, a will occur; If P(A)=0, a will not happen. Shannon defines the self-information of event A as

When p (a) > 0, I(A)=-lg(P(A)).

When P (a) = 0, I(A)=0.

Logarithm is used to simplify the calculation, because if it is directly expressed by probability and the total information of several messages (or multiple events in a system) is obtained by multiplication, the logarithm can be multiplied and summed. Because the probability of a random event is always less than 1, and its logarithm is negative, a negative sign is added before the logarithm of the probability to make the information amount take a positive value.

As can be seen from the above formula, small probability events have more information. This is well understood in geological prospecting: in an area with m units, geological marker A appears on n 1 unit, geological marker B appears on n2 units, geological marker B only appears on units with ore deposits or mineralization, and geological marker A only appears on units with ore deposits. Obviously, N2 > n 1, so (N2/ m) > (n 1/ m). When m, n 1 and n2 are all large, the probability of b = N2/m, and the probability of a = N 1/m ... Define the formula according to its own information.

I(A)=-LG(P(A)=-LG(n 1/m)= lgm-lgn 1

I(B)=-LG(P(B)=-LG(N2/ m) =lgm-lgn2.

Because m > N2 > n 1, so

I(A)-I(B)=lgn2-lgn 1>0

That is to say, the appearance of geological marker A provides more information about the existence of the deposit than the appearance of geological marker B. ..

Therefore, the amount of information defined in this way meets the requirements of defining information as the content of uncertain things in the message (that is, things that were not known before).

For a group of events S (including E 1, E2, …, En***n events), let the probability of the occurrence of event Ek be P(Ek)=Pk, and define the statistical average of this group of events' own information as follows.

Information theory, system theory and geological prospecting work

In the formula, lgPk=0 when Pk=0. Usually, the statistical average of S's self-information is called the entropy of S.

Mutual information and joint entropy can be defined for two groups of events. Set theory in mathematics will be used at this time, so it will not be described. Interested readers can refer to the relevant monograph [1].

Fourthly, the development of information theory.

After the birth of narrow sense information theory, it has been greatly developed due to its popularization and application. Its development has roughly experienced three periods:

1.50s was the period when information theory impacted various disciplines. The achievements of information theory have brought unexpected hope to many disciplines. People try to use the concept and method of information to solve many unsolved problems faced by this discipline; This paper attempts to apply information theory to solve problems such as semantics, hearing, nerves, physiology and psychology. For example, the third information theory conference held in London from 65438 to 0955 covered a wide range of topics, including anatomy, animal health, anthropology, computers, economics, electronics, linguistics, mathematics, neurophysiology, neuropsychiatry, philosophy, phonetics, physics, political theory, psychology and statistics. However, the narrow sense of information theory has some limitations, such as not considering the meaning of information (such as whether the information is true or not) and the value of information, and not describing fuzzy information, so few achievements have been made in this regard.

2. In 1960s, information theory was a period of digestion and understanding, and a period of major construction on the existing basis. The research focuses on information and source coding, noise theory, signal filtering and prediction, modulation and information processing. These are the so-called generalized information theories, which mainly study communication problems and also involve psychology.

Since 1970s, information theory has developed into generalized information theory or information science. This development of information theory is related to the worldwide new technological revolution. In the mid-1970s, Alvin Toffer toffler, an American futurist, wrote The Impact of the Future to discuss the future development of human society. 1980 released the third wave. He believes that the first wave was the agricultural revolution from 8000 to 1000 years ago, the second wave was the industrial revolution in the middle of 18 century, and the third wave was the "information revolution" that began in the 1940s. Therefore, people are more and more aware of the importance of information, and realize that information can be fully utilized and enjoyed like matter and energy. The concepts and methods of information theory have penetrated into various scientific fields, and it is urgent to break through the narrow scope of narrow information theory and make it the basic theory of information problems encountered in various human activities, thus promoting the further development of many other emerging disciplines. Since 1970s, many scholars have put forward the concepts of effective information, generalized effective information, semantic information, probabilistic information and fuzzy information. At present, people have widely applied the established laws and theories of information to physics, chemistry, biology, psychology and management. An information science that studies the generation, acquisition, transformation, transmission, storage, processing, display, identification and utilization of information, that is, generalized information theory, is taking shape. In the following chapters of this chapter, some important concepts in generalized information theory will be described in combination with the characteristics of geological prospecting.