konstantinoscs / web_crawler

A collection of a simple web creator and server and a web crawler

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

My final Systems Programming project

Webcreator.sh: A shell script that creates a number of websites

Server: A simple web server that only accept GET requests. Responds with 200, 403, 404 or 500

Crawler: A web crawler that tries to download all the pages from the server by taking a starting page and then recursively downloading all links found in it.

About

A collection of a simple web creator and server and a web crawler


Languages

Language:C 95.4%Language:Shell 3.8%Language:Makefile 0.9%