Low Cost Journal,International Peer Reviewed and Refereed Journals,Fast Paper Publication approved journal IJEDR(ISSN 2321-9939)
apply for ugc care approved journal, UGC Approved Journal, ugc approved journal, ugc approved list of journal, ugc care journal, care journal, UGC-CARE list, New UGC-CARE Reference List, UGC CARE Journals, ugc care list of journal, ugc care list 2020, ugc care approved journal, ugc care list 2020, new ugc approved journal in 2020,
Low cost research journal, Online international research journal, Peer-reviewed, and Refereed Journals, scholarly journals, impact factor 7.37 (Calculate by google scholar and Semantic Scholar | AI-Powered Research Tool)
The World Wide Web is interlinked collection of billions of documents formatted using HTML. Web Crawler continuously keep on crawling the web are finding any new page web pages that have been added to the web, pages have been removed from the web. Web Crawler created by Brian Pinkerton of the University of Washington and launched on April 20 1994, Web Crawler was the first Search Engine that was powered by web crawler. The crawler is a program that visit web sites and read their pages and other information in order to create entries for the search engine index. which is also known as a “Spider” or a “boot”. The main purpose is Web Crawling the data is indexed by the search engine. All the main search engine, such as Google and Yahoo, use spider to build and revise their indexes. Moreover, the spider is used for automatic maintenance tasks on the web sites.
Keywords- bots, seeds, spider, crawler.
Unique Identification Number - IJEDR2101033Page Number(s) - 237-240Pubished in - Volume 9 | Issue 1 | March 2021DOI (Digital Object Identifier) -    Publisher - IJEDR (ISSN - 2321-9939)
Cite this Article
Akash Chauhan,   "Web Crawling"
, International Journal of Engineering Development and Research (IJEDR), ISSN:2321-9939, Volume.9, Issue 1, pp.237-240, March 2021, Available at :http://www.ijedr.org/papers/IJEDR2101033.pdf