Stage 1: JavaSE basic core
1, deeply understand Java object-oriented thought.
2. Master the basic API commonly used in development.
3. Skillfully use collection framework, IO flow and exception.
4. It can be developed based on JDK8.
Phase 2: Hadoop ecosystem architecture
1, installation and operation of Linux system
2. Master the syntax of Shell scripts.
3. Use of development tools such as 3.Idea and Maven.
4. In-depth analysis of 4.Hadoop composition, installation, architecture and source code, and skillful use of API.
5.Hive installation and deployment, internal architecture, skilled use of its development needs and enterprise-level optimization.
6. The internal principle of 6.Zookeeper, the election mechanism, and the response under the big data ecology.
Stage 3: Spark ecosystem architecture
1, the initial installation and deployment of Spark, proficient in using the basic API of Spark kernel, advanced RDD programming, mastering accumulators and broadcast variables, mastering Spark SQL programming and how to define functions, detailed explanation of Spark kernel source code (including deployment, startup, task division and scheduling, memory management, etc.), and Spark's enterprise-level tuning strategy.
2. Install and deploy DophineScheduler, and skillfully use workflow scheduling and execution.
3. Understand the modeling theory of data warehouse, fully familiar with the data analysis index system of e-commerce industry, quickly master a variety of big data technology frameworks, and understand a variety of data warehouse technology modules.
4. The deployment and use of 4.HBase and Phoenix, the explanation of principle architecture and enterprise-level optimization.
5. Development tool git &；; Clever use of Git Hub
6.Redis introduction, basic configuration description, jedis mastery.
7. Introduction, installation, deployment and optimization of 7.ElasticSearch
8. Fully understand the construction and use of the user portrait management platform, the design idea of the user portrait system, and the design process and application of the tag, and initially understand the machine learning algorithm.
9. Build a fully functional enterprise-level offline data warehouse project independently, improve the actual development ability, strengthen the understanding and cognition of each functional module of offline data warehouse, realize the actual needs of various enterprises, and accumulate experience in project performance tuning.
Phase 4: Flink ecosystem architecture
1, master the basic architecture of Flink and the idea of stream data processing, use many Soure and Sink of Flink to process data, and use basic API, Window API, state function, Flink SQL and Flink CEP to process complex events.
2. Use Flink to build a real-time warehouse inventory project, and skillfully use Flink framework to analyze and calculate various indicators.
3, ClickHouse installation, use and optimization
4. Project actual combat. Close to the actual processing scene of big data, multi-dimensional design of actual combat projects can help us master the solutions of big data needs more widely, participate in the whole process of project construction, improve students' actual combat level in a short time, strengthen their understanding of various commonly used frameworks, and quickly accumulate actual combat experience.
5. You can choose master's recommendation and machine learning projects, and be familiar with and use system filtering algorithms and content-based recommendation algorithms.
6. Reconstruct the e-commerce project with the complete set of big data products of Alibaba Cloud platform, and be familiar with the Alibaba Cloud solution of offline warehouse inventory and real-time indicators.
The fifth stage: employment guidance
1, from the perspective of technology and project according to the enterprise interview,
2. Be familiar with the use of CDH in production environment.
3. Resume guidance
The above is what big data training needs to master. Of course, you can also try to teach yourself.