Kanjiiro / ImgScraper

A python script to scrape images form a website

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ImgScraper

Simple image scraper for a simple site.

TODO

  • Use one function for the multiprocess
  • Removed the use of multiple files splitted in bash
  • Gets number of total processes and uses the 80% of the total processes for the downloader
  • Duplicate name control
  • Img Sub folder tree with sub-tree for bodyparts
  • Add support for different url's
  • Hash image comparison for finding dupes

About

A python script to scrape images form a website

License:MIT License


Languages

Language:Python 95.9%Language:Shell 4.1%