1, name: also known as web spider, web robot.

2. Explanation: It is a program or script that automatically crawls information from the World Wide Web according to certain rules.

3. Background: With the rapid development of the network, the World Wide Web has become the carrier of a large amount of information, and how to effectively extract and use this information has become a huge challenge.

Second, function

1. Description or definition of the grab target.

2. Analysis and filtering of web pages or data.

3. Search strategy for web pages or data.

Third, classification

1, universal web crawler.

2. Focus on web crawler.

3. Incremental web crawler.