Linux: Because big data-related software runs on Linux, we should learn Linux well. Learning Linux well is of great help for you to quickly master big data related technologies, and can help you better understand the running environment and network environment configuration of big data software such as hadoop, hive, hbase and spark. , let you step on a lot of pits less, learn to understand scripts, and make it easier for you to understand and configure big data clusters.
Hadoop: This is a popular big data processing platform, which is almost synonymous with big data, so it is necessary.
City zoo: It's a panacea. It will be used when installing Hadoop's HA, and it will also be used in future Hbase.
Mysql: We have finished learning the processing of big data, and then we have to learn the mysql database, a processing tool for small data, because it will be used when installing hive later. What level does mysql need to master? You can install it on Linux, run it, configure simple permissions, modify the password of root, and create a database.
Sqoop: Used to import data from Mysql into Hadoop.
Hive: This thing is an artifact of people who understand SQL syntax. It allows you to handle big data easily.
Oozie: Now that you have learned Hive, I'm sure you need it. It can help you manage your Hive or MapReduce or Spark scripts and check whether your program is executed correctly.
Hbase: This is the NOSQL database in Hadoop ecosystem. Its data is stored in the form of keys and values. Keys are unique, so they can be used to copy data. Compared with MYSQL, it can store more data.
Kafka: This is a good queuing tool.
Spark: used to make up for the lack of data processing speed based on MapReduce.