Pringlez / Web-Crawler

A Web Crawler - Java

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Web-Crawler

Java SE Project

Project Details

The project's main goal is to allow users scan or scrape websites for underlying details embedded in web pages. The application will employ the use of Java 1.8 SE Edition. A GUI application will sit on top of the web crawler which will allow users to quickly add URLs to a processing queue. This queue will allow users to scan multiple websites and addresses simultaneously. Usually a computer with two or more processor cores will process this queue fairly quickly. Maven will also be employed to manage the dependencies the application needs to function correctly. Over time I'll write the java docs and tests as the project gets larger.

TODO

  • Issue - Timeout error handling
  • ToDo - List scroll function needs implementing
  • ToDo - Write tests
  • ToDo - Add more URL details
  • ToDo - Write JavaDocs
  • ToDo - Write tests for Maven

About

A Web Crawler - Java

License:Apache License 2.0


Languages

Language:Java 100.0%