NSCoder / pychromeless

Python Lambda Chrome Automation (naming pending)

Home Page:https://engineering.21buttons.com/crawling-thousands-of-products-using-aws-lambda-80332e259de1

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

PyChromeless

Python (selenium) Lambda Chromium Automation (naming pending)

PyChromeless allows to automate actions to any webpage from AWS Lambda. The aim of this project is to provide the scaffolding for future robot implementations.

But... how?

All the process is explained here. Technologies used are:

Requirements

Install docker and dependencies:

Working locally

To make local development easy, you can use the included docker-compose. Have a look at the example in lambda_function.py: it looks up “21 buttons” on Google and prints the first result.

Run it with: make docker-run

Building and uploading the distributable package

Everything is summarized into a simple Makefile so use:

  • make build-lambda-package
  • Upload the build.zip resulting file to your AWS Lambda function
  • Set Lambda environment variables (same values as in docker-compose.yml)
    • PYTHONPATH=/var/task/src:/var/task/lib
    • PATH=/var/task/bin
  • Adjust lambda function parameters to match your necessities, for the given example:
    • Timeout: +10 seconds
    • Memory: + 250MB

Shouts to

Contributors

About

Python Lambda Chrome Automation (naming pending)

https://engineering.21buttons.com/crawling-thousands-of-products-using-aws-lambda-80332e259de1

License:Apache License 2.0


Languages

Language:Python 80.6%Language:Makefile 19.4%