Not a real CTF
Scraper CTF defied all odds, why need a webrowser in the first place! Ctfs can easily be hosted over netcat!
beware of Python2.7, hosters!
4 D3SCR1PT10N Y0U D0N7 W4NN4 R34D
since the ancient times, since when DDOS has existed, since when someone came up with the idea of CTFs...
CTFers have shown stupidity and will continue... they will waste atleast 1 hour 17 minutes and 36 seconds during a CTF, downloading files and then moving them around their file system ,., sayS NOT ME! BUT A 2019 statistics by @invalid-user in discord! This person took live samples during the following CTFs... ABCCTF ( A BIG CTF CTF), 1337x99999CTF, R-18CTF and CTFNASA!
Notable results kudos to him:
87.51% of CTfeRS click on the file to download and then access it from their Downloads folder!
39.29%, yes @naughtyb0y I am pointing at you! don't even make a folder DEDICATED for the CTF!
6.94% actually use
wget, CHEERS!
26.11% n00bs create folders like CTF_LOVECON_2K20, can you believe it folks????
15.73% use
curland fail, ^wtf^
(results are not-independent; venn diagrams predicted to be messy; contact - prof. ocip)
The real stuff
So, to solve the above mentioned problem, I bring to you the Ultimate CTF Vaccuum Cleaner or was it Scraper? Anyway... if you weren't offended by any of this feel free to continue...
What do we have going here?
--> this cli will helps scrape all the challenges in your favorite CTF and organize them into your defined folder as per the challenge categories!
--> you have the control of downloading all or none, or category-wise, the files that come with a challenges! If you choose None, you will have all the challenge description atleast with you!
--> Scraping is just the tip of things, we have also provided a free FLAG Submitter for our open-source customers! If you are not open-source, then what are you doing here? *=?
--> Want more? Well, how about getting rid of the hassle of logging in again and again with new session saves! Register in one time and forget about the website entirely... hmm, sounds euphoric!
--> currently supports CTFd, CTFx and other platforms coming soon! Mostly because there aren't any to test on!
DE-BENEFACTO
need motivation for cloning this repo? read below:
--> It will save time (duh 🥴), especially when CTFs can been time-clinching sometimes!
--> CTFs generally update with the challenge description and files, and also add in new challeneges! Rather that scrounging through the slow website or missing an update notification, just run the scraper; it will update the files!!
--> With a good Internet connection, you can scrape everything within 10 seconds 🤯
--> Make solving CTFs almost offline! Also many a time, flag-submitting is a hassle! The java-script behind responds weirdly and you just sit there not knowing whether your flag got submitted or not!
--> Help keep all the files in an organized manner, make write-ups or uploading the files to github!
--> can save multiple sessions for a single-ctf or muliple ctfs at the same time; can also be used for a very specific purpose if you know what I mean 😏!
All you care about!
git clone the repo using,
git clone https://github.com/Masrt200/Scraper-CTF
then follow these commands!
cd Scraper-CTF
pip install -r requirements.txt
check python3 scraper.py -v
if runs, you are good to go!
P.S.
Consider sharing any opinions or updates to this project! Also do report if there are any issues!