kordless / crawlapart

System for serving crawlapart.com, your favorite bookmarking price monitor. Forking from https://gitlab.com/ch7ck/infomat/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

SmallCrawl

This repo contains a couple of things that you should use to crawl a site with a Mozilla binary as well as a GeckoDriver Process running.

The idea with all of this code is to be able to pretend that you're a browser.

Gettin' Started

Requirements to run:

Tool Link
Python Port of Webdriver webdriver geckodriver python wrapper
GeckoDriverBinary Releases Page
Mozilla Profile Needs Link

BrowserSession.py needs to be imported. Take a look at BrowserSession.py.

You will need geckodriver running locally:

geckodriver -vv --marionette-port 2928

To run this you will need the W3c webdriver geckodriver python wrapper.

Lookup Word

Lookup word's primary purpose is to ensure that you can lookup a word on google. This commit should include some simple flags to allow a plugin to interact with lookupword and save a screenshot of the whole webpage.

About

System for serving crawlapart.com, your favorite bookmarking price monitor. Forking from https://gitlab.com/ch7ck/infomat/


Languages

Language:Python 99.2%Language:Dockerfile 0.8%