Rand Fishkin at SEOMoz Blog presents an overview of search engine crawling and indexing in this 10 min video.
“Crawling” (or spidering) refers to the process of a computer program (Web crawler, Web spider, or Web robot) that browses the World Wide Web in a methodical, automated manner to maintain up-to-date information.
Search engine “indexing” refers to the collecting, parsing, and storage of the data that is crawled to facilitate fast and accurate information retrieval for searchers (you and I).
In this video Rand presents the basics of crawling and indexing for the purpose of understanding how search engine optimization can enhance the presentation of a website’s visibility to searchers.
Search Engine Crawling and Indexing
- by George Alger