skrp / archive.org

mass scrape archive.org by collection

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

#######################################
# archive.org mass scrape by collection
           TWO STEPS TO SIMPLE QUE
# 1 ###################################
Output item identifiers of collection into que
# 2 ###################################
[A - Fetch all identifier htmls ]  
[B - grep interested encodings ]
[C - output download links by encoding ]

ENCODINGS:
  MP4
  MP3
  PDF
  TXT
  JPG
  PNG
  GIF
  TIF
  META

(( feeds 'get' api     https://raw.githubusercontent.com/skrp/HIVE/master/DEMON ))

About

mass scrape archive.org by collection


Languages

Language:Perl 68.4%Language:Shell 31.6%