mayankagnihotri7 / data-crawler

IMDB data scrapper for scrapping the top 100 movies from the website.

Data-crawler

Getting started

• Clone the repo using 'https://github.com/maxagno3/data-crawler.git'.
• Install the dependencies by going into respective folders using 'yarn install' or if you don't have yarn installed on your machine then delete the package-lock.json from the respective folders and install dependencies using 'npm i' or 'npm install'.
• Start the server by going into the client or server folder or both and use 'yarn start' or 'npm start'.

Routes

Movie list can be seen on http://localhost:3000/
Route for the API is http://localhost:4000/
For Scrapping data from IMDB's website, you can enter http://localhost:4000/movie url on the browser, it will scrape data of 100 top movies and will save it into a JSON file.

Library Used

• Axios ( https://www.npmjs.com/package/axios ).
• Cheerio ( https://www.npmjs.com/package/cheerio ).
• TailwindCSS ( https://tailwindcss.com/ ).

Developer

Mayank Agnihotri

• Twitter: https://twitter.com/max_AgNO3
• LinkedIn: https://www.linkedin.com/in/mayank-agnihotri/