After reading "The Age of Big Data", I feel that an era of great change is coming. Although it is not clear what way of thinking and operation should be completely changed, it is obvious that the author wants to. End? Or subvert some theories, methods and ways that are traditionally considered as the basis of our thinking and existence. In the face of such an idea, my mind was strongly shocked and I couldn't help shivering.

? In the era of small data, we will assume how the world works, and then verify this assumption by collecting and analyzing data. With the transition from the imaginary era to the data era, we may also think that we no longer need theory. ? The book will almost certainly subvert the theory and method of statistics, and try to quote Anderson, editor-in-chief of Wired magazine. The theory of quantum physics has been divorced from reality? Coming? End? Quantum mechanics. I am very happy about this, because statistics and quantum mechanics are both subjects that I failed in college. But these two theories are too big, too authoritative and too basic. I don't think I can get rid of these two things that have been a headache for my whole life by a book. In fact, the author dare not put forward a clear argument to subvert them. After all, he added it to the front. Probably think? Such an umbrella.

In recent decades, we have always come across all kinds of new ideas. In the face of new thinking, the first thing we should do is to break the tradition and keep pace with the times. Even if the brain can't keep up, the mouth should keep up, otherwise it may be labeled as rigid and even hinder the development of the world. Since big data is? Inevitable changes in the future? So I have to? Not limited by the traditional mode of thinking and the inherent prejudice implied in a specific field? Let's join the author in denying statistics and quantum mechanics. I don't like them anyway, and I can't learn.

When our human data collection and processing ability reaches Pb or even greater, we can turn the samples into all. After we have the ability to face up to heterozygosity and ignore accuracy, it seems that we can really abandon the statistics based on sample surveys. But through statistics and quantum mechanics and many other methods? What we may think is no longer needed. Theoretically, they are almost all based on the same basic logic. If you accidentally give logic or logical thinking or logical reasoning together? No longer needed? If so, I'm worried!

The era of big data, 16 pages? The core of big data is prediction? . Does logic describe spatio-temporal information? Class? With what? Class? Long-term effective and unchangeable order change relationship rules. They seem to be doing the same thing. But big data wants it? Not causality, but correlation? ,? It's enough to know what it is, not why. And the sufficient reason law in the four basic laws of logic (identity, contradiction, law of excluded middle and sufficient reason law)? Clear definition? Everything has a reason to exist. The three parts of logical reasoning-inductive logic, retrospective logic and deductive logic-are all based on causality. The two seem to be opposite. There should be only one opposite result of two methods on the same thing, that is, denying one of them. This is exactly what I'm worried about.

But I can't wait and see, waiting for which one looks like a bystander? Stand out? Because I'm inside. If the problem is not solved, I can't think and work, and naturally I can't live! Besides, there are two more terrible things.

First, quantum mechanics has been practiced for more than one hundred years. In order to deal with the problem of hybridization, quality and speed have been combined with energy. In order to reconcile the contradiction between quantum mechanics and relativity, quantum field theory was developed, and then wormholes and Rosen bridges were created. Finally, four-dimensional space-time has been bent into a way to allow time travel, and I can't wait to build that terrible time travel machine at once. What's the only way to stop this? Einstein? Children? Fooling around? This is a causal relationship, because the father is the father and the son is the son. So will big data create a time machine by facing up to hybridity and giving up the law of causality, so that dad is no longer a father and son is no longer a son? Second, the fundamental difference between man and machine is that man has logical thinking and machine doesn't. Is the "big data era" also worrying? It will be the machine, not the human, that will make the final decision? . If that day was really because I gave up logical thinking and the machines described in science fiction movies ruled the world and destroyed mankind, then I might as well jump off the building now.

Fortunately, I know that I am a layman in statistics, quantum mechanics, logic and big data. Perhaps the above articles are all nonsense, and the so-called worry does not exist at all. But when problems arise, it's better to solve them, otherwise I can't sleep. If you can't solve it yourself, you can only rely on experts to point out the maze.

So I want to give the author of Big Data Age a reasonable suggestion: Continue to write this book, and at least add a logical thinking in the fourth part of Big Data Age.

Experience of Reading Big Data Times (2)

With the advent of the information age, we feel that technological changes are changing with each passing day, which is followed by changes in lifestyle. The information age we are commenting on is a thing of the past. Nowadays, the era of big data has become a hot topic. The author explains information and data here, just to explain the connection and difference between information and data first, and also to explain why the information age has become the era of big data. What does the era of big data bring us?

Definition of information and data. Wikipedia explains that information, also known as information, is a highly generalized abstract concept, a developing dynamic category, and the contents and names to be exchanged with each other. There is no unified definition of information, but it is well known that information is objective, dynamic, transitive, enjoyable and economical. Data: or data, refers to the symbolic records describing things, which can be defined as meaningful entities, and it involves the existing forms of things. It is a discrete and objective description of a group of events, and it is the original material that constitutes information and knowledge. Data can be divided into analog data and digital data. Data refers to computer processing? Raw materials? , such as graphics, sounds, words, numbers, characters and symbols. As the name implies, data is the original virgin land, which needs to be reclaimed. Information is processed and can be disseminated. The information age depends on the outbreak of data, but when the data broke out of control, the era of big data came into being. Is this the background material that the book "The Age of Big Data" has not elaborated?

In the book Big Data Era, the difference between Big Data Era and Small Data Era: 1, thinking convention. The difference and transformation in the era of big data is to give up the desire for causality and pay attention to relevance instead. So as long as you know? What is this? Without knowing it? Why? . The author's language is absolute, but he reflects on its essential differences. There are more and more miscellaneous data, which leads to the application idea can only be observed as much as possible, instead of reasoning with all its resources? This is also a wise move. 2. use. Small data stays in explaining the past, and big data uses the past to predict the future. The author thinks that the purpose of data has nothing to do with the data itself, but with the interpreter of the data, and correlation is more conducive to predicting the future. 3. structure. Big data is more reflected in the integration and processing of massive unstructured data itself. Big data is more like theory and reality go hand in hand. Theoretical creation methods deal with unstructured data, and the results are verified by the future. 4. Analysis basis. Big data is a process from quantitative change to qualitative change in the context of the Internet. The author believes that the era of small data, that is, the information age, is the premise of the era of big data. The era of big data is sublimation and evolution, and the essence is complementary, not mutually exclusive.

The story of the future of data. What expectations and inspirations does the development of data bring to us? The banking industry naturally has the potential of big data. Massive data such as customer data, transaction data and management data are constantly growing, and massive opportunities and challenges follow, adapting to changes and eliminating the fittest. We can have broader business development space, more accurate decision-making ability and better management ability, all of which are based on the ability of data collection, collation, control and analysis, as well as innovative thinking and execution. So, architecture? Data warehouse? , cultivate? Data thinking? , development? Data governance? , create? Data fusion? , realize? Data application? Hug? Big data? Times, grab value from data, laugh at change, and win the future.

Experience of Reading Big Data Times (3)

This book mainly introduces the application of big data in modern business operations and its impact on modern business operations.

The structural framework of The Age of Big Data follows the general way of academic books. That is, starting from the phenomenon, and then explaining this phenomenon through anatomy. Then I predict the future through explanation, and put forward my own views and countermeasures for possible problems in the future.

Let's focus on the main contents of the book "The Age of Big Data".

At the beginning of the era of big data, Google successfully predicted the outbreak and spread direction of H 1N 1 in the United States in 20XX, as well as possible potential patients. Google's forecast will be nearly a month ahead of the government's, in contrast, the government can only get the relevant data one or two weeks after the flu outbreak. At the same time, the correlation between Google's forecast and government data is as high as 97%, which means that the confidence interval of Google's forecast data is 3%, which is far less than the conventional confidence interval of 5% in traditional statistics! And this figure is the best proof of the relative accuracy of prediction results and the predictability of events in the era of big data! Through such and such cases, Victor proposed that in the era of big data? Sample = population? The idea. We all know that when the sample is infinitely close to the population, the calculated descriptive data will be infinitely close to the nature of the event itself. What was taken before? sample

Next, Victor passed the failure of IBM's pursuit of high-precision computer translation plan and Google only scanned and stored all the corresponding text sentences in the thesaurus, so no matter what needs to be translated, as long as it is related to Google thesaurus, there will be translation. Although sometimes translation is meaningless, it is correct most of the time. Therefore, the success of Google's computer translation program shows that the pursuit of accuracy in the era of big data is not particularly obvious. On the contrary, the era of big data is based on big data, so the era of big data pursues all-round coverage of digital measurement, no matter how accurate, because a large amount of data will bury the influence of a few problematic data. At the same time, a large amount of data will infinitely approach the true colors of things.

Later, Victor predicted that an important professional data scientist was born in the era of big data. This scientist is a combination of mathematicians, statisticians and programmers. This group of people will be able to get any results they want from the obtained data. In other words, as long as there is enough data, all the external and internal things we don't want others to know will be displayed in front of these guys. Therefore, in order to prevent personal privacy from being used by this group of people in the era of big data, Victor suggested that this group of people be divided into two parts. One part is to use data to serve the business sector, and the other part is responsible for reviewing whether these people legally obtain and apply data and infringe on personal privacy.

In any case, the era of big data will come, whether we accept it or not!

I think the book Big Data Age is well written and worth reading. Because it will give us a lot of inspiration, such as your comments or photos on related social networking sites are likely to be? Data scientist? Users use it and then sell the relevant data to major online stores. However, the fact is that we will be tempted by prophecy. So, be careful what you leave online.

I like this book because it shows me a new world.

Experience of Reading Big Data Times (4)

I used the weekend to read Tu Zipei's masterpiece Big Data in one breath. This book is very good-looking, fluent and fascinating. In the book, what you read is not big data technology, but more about the evolution of American politics, economy, society and culture related to big data. As an information practitioner, after reading the whole book, I deeply felt the respective characteristics of information technology in China and the United States, and also saw the gap between us and the United States. There are several aspects of experience, but you can basically see the whole picture at a glance.

First, the breadth and depth of the disclosure of government business databases. In recent years, with the advancement of information disclosure in China, governments at all levels are actively promoting online government information disclosure through the construction of government portals. At present, our information disclosure is mainly the disclosure of government policies, laws, regulations, standards, official documents, job responsibilities, work guidelines, work trends, personnel appointment and dismissal and other administrative affairs. Of course, the real-time disclosure of government business databases has also made great progress. In the portal of China government, you can query some public welfare databases, such as the economic statistics of the National Bureau of Statistics, the national air and hydrological data provided by the data center of the Ministry of Environmental Protection, the national meteorological data provided by the General Administration of Meteorology, and the national flight information provided by the Civil Aviation Administration. Many business data can also be found on the websites of ministries and commissions, such as the project approval database of the National Development and Reform Commission, the enterprise credit database of the Industrial and Commercial Bureau, the land certificate database of the Ministry of Land and Resources, the coal mine safety early warning information database of the State Administration of Work Safety, and the bidding information database of various projects. This is a great progress, and it is also the achievement and value of e-government construction for so many years! However, a lot of data in the government business database has not been made public at present, and many data are due to departmental interests and? Secret? And other factors, but also limited to internal personnel use, not public; The published data are limited to some basic information and statistical information, and more data are not published. Judging from the practice of American data disclosure recorded in Big Data, the breadth and depth of American data disclosure are relatively large. Americans think? Data collected with taxpayers' money should be provided to taxpayers free of charge? Although the US government is actually opposed to data disclosure, the wishes of the people cannot be violated. American government's business data is becoming more and more open, especially after the Obama administration signed the document of "transparent and open government". DATA.GOV is the newly-built unified data open portal of american league government. The website organizes all kinds of open data according to original data, geographic data and data application tools, and has accumulated 378,529 original and geographic data sets. There is no such website with open data in China. In addition, due to different systems, the disclosure of business information in the United States is also very in-depth. For example, the President of the United States makes it public online? White House visitor records? Announced all kinds of information about people even visiting the White House; The FedSpending website in the United States can track, record and analyze every financial expenditure of the federal government one by one. This should not be realized in China at present.

The second is the analysis of government business data. At present, the business data provided by government websites at all levels in China are basically data tables, and some websites can provide some statistical charts, but rarely can cross-departmental online analysis and data association analysis be realized. This is mainly because China's government informatization construction is still in the stage of departmental construction. The United States is moving faster in this regard. American DATA.GOV website not only provides raw data and geographic data, but also provides many data tools, many of which are provided by the public, non-profit organizations and some commercial organizations. These applications provide data processing, online analysis, association analysis based on social networks and other means. For example, the White House visitor search tool provided on DATA.GOV can search visitors' information and associate White House visitors with other Weibo and social networking sites to improve the transparency of visitors.

The third is about the privacy of personal data. In America, citizens' privacy and ownership are inviolable. There is no personal ID card in the United States, so it is impossible to establish personal information association based on personal ID number. Central database? This proposal has also been rejected many times. This is not a problem in China. Every citizen has unique identity information, and the basic information of citizens can be obtained through ID card information. In the future, with the construction of national population database and other basic resource banks, citizens' social security, medical care and other related information can also be easily obtained. Of course, information is still limited to government departments, but it is difficult to completely guarantee that these integrated personal information will not be leaked or used.

Data is the foundation of information construction. Learning from each other and learning from each other in the field of big data will push the world into the information age. I'm glad to see that the U.S. government started in 20XX? Big data research and development plan? Invested 200 million US dollars to promote research in the fields of big data extraction, storage, analysis, sharing and visualization, which is comparable to supercomputing and Internet investment. In the same year, China municipal government also approved it in 20XX? National government informatization construction project planning in the 12th Five-Year Plan? The total investment is expected to be tens of billions, and there are five major construction projects: population, legal person, space, macroeconomics and culture. The era of open, accessible and intelligent big data has arrived!

I recommend it carefully.