arnabpal2022 / web-crawler

Embark on an adventure into the depths of the web with this powerful web crawler made with Node, Axios and Cheerio.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

web-crawler

A Basic Web-Crawler Attempted in Node.js (Libraries used axios and cheerio). The Project demonstrates core web crawling functionalities:

  • fetching
  • parsing
  • link extraction
  • iterative link following
  • crawl limit control
  • success-error ratio calculation.

(working in progress)

About

Embark on an adventure into the depths of the web with this powerful web crawler made with Node, Axios and Cheerio.

License:MIT License


Languages

Language:JavaScript 100.0%