aws-glue

There are 9 repositories under aws-glue topic.

• aws-sdk-pandas

  aws / aws-sdk-pandas

  pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

  pythonawspandasapache-arrowapache-parquetdata-engineeringetldata-scienceredshiftathenalambdaaws-lambdaaws-glueemramazon-athenaglue-catalogmysqlamazon-sagemaker-notebookmodinray
  Language:Python 4073
• awesome-aws-workshops

  dgomesbr / awesome-aws-workshops

  (Unofficial) curated list of awesome workshops found around in the internet. As we all have been there, finding that workshop that you have just attended shouldn't be hard. The idea is to provide an easy central repository, in a collaborative way.

  aws-workshopsserverlessaws-iotapplication-modernizationsql-serveranalyticsamazon-textractamazon-sagemakersagemaker-workshopawsaws-iamimmersion-dayamazon-sagemaker-workshopaws-glueamazon-eks-workshopiot-aws
  Language:HTML 413

• tokern / piicatcher

  Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

  aws-athenaaws-glueaws-redshiftcatalogdatadata-catalogdatabasephipiipythonsnowflake
  Language:Python 322
• jikkou

  streamthoughts / jikkou

  The Open source Resource as Code framework for Apache Kafka. Jikkou helps you implement GitOps for Kafka at scale!

  apache-kafkaautomationaws-gluecluster-managerdatameshdevopsgitopshacktoberfestinfrastructure-as-codejavakafkakafka-clusterkafka-managerkafka-topicyaml
  Language:Java 253
• dataall

  data-dot-all / dataall

  A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

  awsaws-glueaws-lake-formationaws-s3datadata-scienceetl-frameworklakeformationlakehouseredshift
  Language:Python 246

• aws-samples / data-lake-as-code

  Data Lake as Code, featuring ChEMBL and OpenTargets

  awsaws-cdkaws-cdk-constructsaws-glueaws-lake-formation
  Language:TypeScript 173

• awslabs / athena-glue-service-logs

  Glue scripts for converting AWS Service Logs for use in Athena

  glue-scriptscloudtrail-logselb-logsathenaglue-jobaws-glues3-log-parsercloudfront-logsalb-logsvpc-flow-logs
  Language:Python 140

• aws-samples / amazon-deequ-glue

  Automated data quality suggestions and analysis with Deequ on AWS Glue

  deequaws-glueawsdata-quality
  Language:Scala 88
• cloud-experiments

  aws-samples / cloud-experiments

  Open innovation with 60 minute cloud experiments on AWS

  aws-gluedata-scienceamazon-athenaamazon-sagemakeramazon-s3machine-learningamazon-comprehendamazon-rekognitionnotebooksaws-cloud
  Language:Jupyter Notebook 87
• streamlit-application-deployment-on-aws

  aws-samples / streamlit-application-deployment-on-aws

  Streamlit EDA Dashboard Powered by AWS Cloud

  streamlit-dashboardawsaws-cloudformationaws-glueaws-athenaaws-cognitoaws-sagemaker
  Language:Python 84

• Ditectrev / Amazon-Web-Services-Certified-AWS-Certified-Machine-Learning-MLS-C01-Practice-Tests-Exams-Question

  ⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

  mls-c01awsmachine-learningapache-sparkamazon-athenaamazon-ec2amazon-emramazon-sagemakeraws-glueaws-certifiedaws-certified-machine-learningamazon-cloudwatchamazon-transcribeaws-batchamazon-comprehendaws-lambdaamazon-s3amazon-textractlinear-regressionneural-network
  73

• aws-samples / aws-glue-jobs-unit-testing

  Demo code to illustrate the execution of PyTest unit test cases for AWS Glue jobs in AWS CodePipeline using AWS CodeBuild projects

  automated-testingawsaws-gluepytest
  Language:Python 48

• aws-samples / analyzing-reddit-sentiment-with-aws

  Learn how to use Kinesis Firehose, AWS Glue, S3, and Amazon Athena by streaming and analyzing reddit comments in realtime. 100-200 level tutorial.

  kinesis-firehosedelivery-streamaws-gluedata-streamamazon-athenadata-lakeredditsentiment-analysissentiment-classificationreal-timeself-learningtutorials
  Language:Python 44

• aws-samples / bring-your-own-data-labs

  Bring your own data Labs: Build a serverless data pipeline based on your own data

  analyticsawslabshands-on-labaws-glueaws-s3aws-quicksight
  Language:HTML 44

• cloudposse / terraform-aws-glue

  Terraform modules for provisioning and managing AWS Glue resources

  awsaws-glueetletl-jobglueworkflow
  Language:HCL 34

• DisasterAWARE / aws-glue-schema-registry-python

  Use the AWS Glue Schema Registry in Python projects.

  awsaws-glueschema-registrykafkaavropython
  Language:Python 34

• aws-samples / transactional-datalake-using-apache-iceberg-on-aws-glue

  Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS

  apache-icebergaws-glueaws-dmsaws-athenaapache-spark
  Language:Python 33

• 1oglop1 / aws-glue-monorepo-style

  Example of AWS Glue Jobs and workflow deployment with terraform in monorepo style. Code here supports the miniseries of articles about AWS Glue and python.

  pythonawsaws-glueserverlessdatascienceterraform
  Language:Python 32

• awslabs / amazon-athena-cross-account-catalog

  🌉 Reference implementation for granting cross-account AWS Glue Data Catalog access from Amazon Athena

  amazon-athenaaws-glue
  Language:Python 30

• aws-samples / monitoring-apache-iceberg-table-metadata-layer

  Sample code to collect Apache Iceberg metrics for table monitoring

  apache-icebergawsaws-cloudwatchaws-glueaws-lambdadata-qualitymonitoringsam-cliapache-sparkpyiceberg
  Language:Python 28

• SWO-GS / athena-cloudtrail-partitioner

  Automate the daily partitioning of your CloudTrail bucket in Athena

  athenaawsaws-athenaaws-gluecloudtrailcloudtrail-logsgluegorillastackpartitioning
  Language:JavaScript 28

• tokern / lakecli

  A CLI to manage and monitor permissions in AWS Lake Formation

  awsaws-glueaws-lake-formationpermissionssql
  Language:Python 26

• vincentclaes / serverless_data_pipeline_example

  Build and Deploy A Serverless Data Pipeline on AWS

  awsaws-glueaws-lambdaaws-s3pythonserverless-framework
  Language:Python 26

• aws-samples / aws-glue-streaming-etl-with-apache-iceberg

  Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3

  aws-glueapache-icebergaws-athenaapache-sparkaws-glue-streaming
  Language:Python 25

• andreichiro / data_engineer_end2end

  End-to-end data engineer project

  ansible-playbookapi-restaws-api-gatewayaws-ecs-fargateaws-glueaws-lambdaaws-s3databricksdbt-clouddockerpandaspysparkpythonsqlterraformvitrinedev
  Language:HTML 23

• chgasparoto / terraform-aws-glue

  Terraform module which creates Glue resources on AWS

  awsaws-glueterraformterraform-modules
  Language:HCL 23

• mikaelahonen-solita / aws-glue-tutorial

  AWS Glue tutorial for data developers.

  sparkpysparkaws-gluetutorial
  Language:Python 23

• moritzkoerber / covid-19-data-engineering-pipeline

  A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.

  awsaws-ecraws-glueaws-lambdaaws-s3dockergreat-expectationspysparksparkapiaws-cdkaws-redshiftaws-cloudformationapache-airflowapache-spark
  Language:Python 23

• webysther / aws-glue-docker

  🐋 Docker image for AWS Glue Spark/Python

  sparkaws-gluedevelopmentdockerdocker-imagedockerfileawscdksamaws-clipython-poetrypytestpythonglue-pysparkdata-engineeringpandasapache-arrowglue-catalogetlaws-glue-docker
  Language:Dockerfile 23

• aws-samples / amazon-rds-export-to-s3-automation

  This repository contains source code for the AWS Database Blog Post Reduce data archiving costs for compliance by automating RDS snapshot exports to Amazon S3

  amazon-athenaamazon-eventbridgeamazon-rdsamazon-s3amazon-snsaws-backupaws-cloudformationaws-glueaws-glue-crawleraws-kmsaws-lambda
  20

• aws-samples / aws-glue-crawler-utilities

  This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS CDK applications.

  aws-glueaws-glue-crawler
  Language:Python 19

• amzn / rheoceros

  Cloud-based AI / ML workflow and data application development framework

  bring-your-own-accountawsaiflowclouddata-scienceevent-basedlow-code-frameworkmachine-learningfeature-engineeringaws-glueaws-emrsagemaker-notebook-instancesagemaker-notebookaws-lambdaserverlesssparkpysparkscala-spark
  Language:Python 17

• jhole89 / aws-glue-sbt-quickstart

  Example of how to set SBT up for local development of AWS Glue Scripts

  sbtapache-sparkaws-gluequickstart
  Language:Scala 16

• jonrau1 / AWS-ComplianceMachineDontStop

  Proof of Value Terraform Scripts to utilize Amazon Web Services (AWS) Security, Identity & Compliance Services to Support your AWS Account Security Posture.

  terraformawsguarddutymoduleslambdapythonremediationautomationkinesis-firehosewafsecurity-hubaws-glueaws-configdevopscloud-securitycomplianceaws-cognitoaws-xraydevsecopssecops
  Language:HCL 16

• vincentclaes / glue-devcontainer

  Glue VSCode devcontainer setup

  awsaws-gluegluevscode
  Language:Python 14
• Data-Engineering-Onboarding-Starter

  wednesday-solutions / Data-Engineering-Onboarding-Starter

  This repository contains a 10 step program to enter the world of Data Engineering

  awsaws-gluedatadata-engineeringetlgluesparkworkflowdata-templatedataenggdata-engg-learningdata-engineering-starterdataengg-template
  Language:Python 14