lxwang42 / dream_market_crawler

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Dream Market Crawler

This project was for crawling the darknet marketplace Dream Market

In later March 2019, after months of DDoS attack, the market operators eventually give up and shut down the market place for good, claiming they will transfer the ownership to a another entity, which is, by the owner, refered to as 'a company'.

The marketplace uses very strict anti DDoS and anti spider policies, if one session requests more than 30 pages in 60 seconds, a captcha img would be triggered, thus halt the crawler's job.

The project employs machine learning to solve captcha imgs to automatically sign up and log into dummy accounts. Although the module has only 60-65% success rate(due to small training sample), it was good enough to get the job done.

About


Languages

Language:Python 100.0%