athifongoqa / xandria-dist-scraper

A scraper to assist in extracting valuable information from internet resources to be hosted by the Xandria platform.

Home Page:https://xandria-scraper-2jytui6ygq-ey.a.run.app

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Xandria Distributed Scraper

The system covered in this repositry is with regards to the scraper component of Xandria.

alt text

Installation

Install requirements:

pip3 install -r requirements.txt

Execute

Run:

celery -A tasks worker
uvicorn main:app --reload

Visit: http://localhost:8000/

About

A scraper to assist in extracting valuable information from internet resources to be hosted by the Xandria platform.

https://xandria-scraper-2jytui6ygq-ey.a.run.app


Languages

Language:Python 95.4%Language:Dockerfile 4.6%