vfrank66 / awsglue-local-dev

A local development setup for AWS Glue, modified from https://github.com/big-data-europe/docker-spark

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Build Status DepShield Badge Codacy Badge

Shoutout to BDE2020

Big thanks for all the setup material provided by https://github.com/big-data-europe/docker-spark.

awsglue-local-dev

A local development setup for AWS Glue, modified from https://github.com/big-data-europe/docker-spark

This will build nightly due to how it pulls the AWS Glue scala .jar and the AWS Glue PyGlue.zip file from an s3 public repo, which is also the same jars included in the AWS Glue environment.

You can use repo to get compile time and spark/aws glue some unit testing coverage which you would not otherwise be able to get through deploying through a expensive AWS Glue DevEndpoint!

Example

About

A local development setup for AWS Glue, modified from https://github.com/big-data-europe/docker-spark

License:MIT License


Languages

Language:Shell 63.8%Language:Dockerfile 36.2%