streamlined2 / WebCrawler

Web crawler application that collects domain statistical information and saves it to database

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

webCrawler

Simple web crawler to collect domain statistical information and save it to DB

Application deployed to Heroku service https://very-simple-web-crawler.herokuapp.com/crawler
DB setup script webCrawler/setup.sql

  • Java 17
  • JEE front controller servlet
  • MVC pattern
  • service, DAO layers
  • JPA transaction programmatic management
  • JPA/Lombok annotations for entity classes
  • Embedded Jetty server
  • PostgreSQL 13.4
  • Freemarker templates
  • Java HttpClient to fetch resources
  • Jsoup to analyze HTML

About

Web crawler application that collects domain statistical information and saves it to database


Languages

Language:Java 92.1%Language:HTML 7.9%