crawler
Just trying out some web crawling.
- Interact with website to obtain parsed results
- Determine which results show be acted upon
- Download file, upload it to other service, delete downloaded file
- Check for completed files from other service and copy to server in appropriate location
- Profit?
Prerequisites
Python 3.x (3.5.3 was used as of this writing) PhantomJS (2.1.1 was used)
Run it
- Configure it by customizing config.json and search.json (they should be self explanatory, if you don't understand them then this probably isn't the program you are looking for)
- Run main.py using Python 3