aws-glue

There are 9 repositories under aws-glue topic.

• aws-sdk-pandas

  aws / aws-sdk-pandas

  pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

  pythonawspandasapache-arrowapache-parquetdata-engineeringetldata-scienceredshiftathenalambdaaws-lambdaaws-glueemramazon-athenaglue-catalogmysqlamazon-sagemaker-notebookmodinray
  Language:Python 3951
• awesome-aws-workshops

  dgomesbr / awesome-aws-workshops

  (Unofficial) curated list of awesome workshops found around in the internet. As we all have been there, finding that workshop that you have just attended shouldn't be hard. The idea is to provide an easy central repository, in a collaborative way.

  amazon-eks-workshopamazon-sagemakeramazon-sagemaker-workshopamazon-textractanalyticsapplication-modernizationawsaws-glueaws-iamaws-iotaws-workshopsimmersion-dayiot-awssagemaker-workshopserverlesssql-server
  Language:HTML 410

• tokern / piicatcher

  Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

  aws-athenaaws-gluepiidatabasedata-catalogphidatacatalogaws-redshiftsnowflakepython
  Language:Python 288
• dataall

  data-dot-all / dataall

  A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

  awsaws-glueaws-lake-formationaws-s3datadata-scienceetl-frameworklakeformationlakehouseredshift
  Language:Python 236

• aws-samples / data-lake-as-code

  Data Lake as Code, featuring ChEMBL and OpenTargets

  awsaws-glueaws-lake-formationaws-cdkaws-cdk-constructs
  Language:TypeScript 166

• awslabs / athena-glue-service-logs

  Glue scripts for converting AWS Service Logs for use in Athena

  glue-scriptscloudtrail-logselb-logsathenaglue-jobaws-glues3-log-parsercloudfront-logsalb-logsvpc-flow-logs
  Language:Python 143
• cloud-experiments

  aws-samples / cloud-experiments

  Open innovation with 60 minute cloud experiments on AWS

  amazon-athenaamazon-comprehendamazon-rekognitionamazon-s3amazon-sagemakeraws-cloudaws-gluedata-sciencemachine-learningnotebooks
  Language:Jupyter Notebook 88

• aws-samples / amazon-deequ-glue

  Automated data quality suggestions and analysis with Deequ on AWS Glue

  deequaws-glueawsdata-quality
  Language:Scala 84
• streamlit-application-deployment-on-aws

  aws-samples / streamlit-application-deployment-on-aws

  Streamlit EDA Dashboard Powered by AWS Cloud

  streamlit-dashboardawsaws-cloudformationaws-glueaws-athenaaws-cognitoaws-sagemaker
  Language:Python 80

• aws-samples / analyzing-reddit-sentiment-with-aws

  Learn how to use Kinesis Firehose, AWS Glue, S3, and Amazon Athena by streaming and analyzing reddit comments in realtime. 100-200 level tutorial.

  kinesis-firehosedelivery-streamaws-gluedata-streamamazon-athenadata-lakeredditsentiment-analysissentiment-classificationreal-timeself-learningtutorials
  Language:Python 44

• Ditectrev / Amazon-Web-Services-Certified-AWS-Certified-Machine-Learning-MLS-C01-Practice-Tests-Exams-Question

  ⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

  mls-c01awsmachine-learningapache-sparkamazon-athenaamazon-ec2amazon-emramazon-sagemakeraws-glueaws-certifiedaws-certified-machine-learningamazon-cloudwatchamazon-transcribeaws-batchamazon-comprehendaws-lambdaamazon-s3amazon-textractlinear-regressionneural-network
  43

• aws-samples / bring-your-own-data-labs

  Bring your own data Labs: Build a serverless data pipeline based on your own data

  analyticsawslabshands-on-labaws-glueaws-s3aws-quicksight
  Language:HTML 42

• aws-samples / aws-glue-jobs-unit-testing

  Demo code to illustrate the execution of PyTest unit test cases for AWS Glue jobs in AWS CodePipeline using AWS CodeBuild projects

  automated-testingawsaws-gluepytest
  Language:Python 41

• DisasterAWARE / aws-glue-schema-registry-python

  Use the AWS Glue Schema Registry in Python projects.

  awsaws-glueschema-registrykafkaavropython
  Language:Python 32

• 1oglop1 / aws-glue-monorepo-style

  Example of AWS Glue Jobs and workflow deployment with terraform in monorepo style. Code here supports the miniseries of articles about AWS Glue and python.

  awsaws-gluedatasciencepythonserverlessterraform
  Language:Python 30

• awslabs / amazon-athena-cross-account-catalog

  🌉 Reference implementation for granting cross-account AWS Glue Data Catalog access from Amazon Athena

  aws-glueamazon-athena
  Language:Python 30

• cloudposse / terraform-aws-glue

  Terraform modules for provisioning and managing AWS Glue resources

  awsaws-glueetletl-jobglueworkflow
  Language:HCL 28

• GorillaStack / athena-cloudtrail-partitioner

  Automate the daily partitioning of your CloudTrail bucket in Athena

  awsathenaaws-athenacloudtrail-logscloudtrailpartitioninggorillastackglueaws-glue
  Language:JavaScript 28

• vincentclaes / serverless_data_pipeline_example

  Build and Deploy A Serverless Data Pipeline on AWS

  serverless-frameworkaws-lambdaaws-s3aws-glueawspython
  Language:Python 27

• aws-samples / transactional-datalake-using-apache-iceberg-on-aws-glue

  Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS

  apache-icebergaws-glueaws-dmsaws-athenaapache-spark
  Language:Python 25

• tokern / lakecli

  A CLI to manage and monitor permissions in AWS Lake Formation

  awsaws-glueaws-lake-formationpermissionssql
  Language:Python 25

• chgasparoto / terraform-aws-glue

  Terraform module which creates Glue resources on AWS

  awsaws-glueterraformterraform-modules
  Language:HCL 23

• mikaelahonen-solita / aws-glue-tutorial

  AWS Glue tutorial for data developers.

  sparkpysparkaws-gluetutorial
  Language:Python 23

• moritzkoerber / covid-19-data-engineering-pipeline

  A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.

  apache-airflowapache-sparkapiawsaws-cdkaws-cloudformationaws-ecraws-glueaws-lambdaaws-redshiftaws-s3dockergreat-expectationspysparkspark
  Language:Python 23

• webysther / aws-glue-docker

  🐋 Docker image for AWS Glue Spark/Python

  sparkaws-gluedevelopmentdockerdocker-imagedockerfileawscdksamaws-clipython-poetrypytestpythonglue-pysparkdata-engineeringpandasapache-arrowglue-catalogetlaws-glue-docker
  Language:Dockerfile 23

• aws-samples / monitoring-apache-iceberg-table-metadata-layer

  Sample code to collect Apache Iceberg metrics for table monitoring

  apache-icebergawsaws-cloudwatchaws-glueaws-lambdadata-qualitymonitoringsam-cliapache-sparkpyiceberg
  Language:Python 22

• aws-samples / aws-glue-streaming-etl-with-apache-iceberg

  Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3

  aws-glueapache-icebergaws-athenaapache-sparkaws-glue-streaming
  Language:Python 20

• aws-samples / aws-glue-crawler-utilities

  This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS CDK applications.

  aws-glueaws-glue-crawler
  Language:Python 19

• andreichiro / data_engineer_end2end

  End-to-end data engineer project

  ansible-playbookapi-restaws-api-gatewayaws-ecs-fargateaws-glueaws-lambdaaws-s3databricksdbt-clouddockerpandaspysparkpythonsqlterraformvitrinedev
  Language:HTML 18

• amzn / rheoceros

  Cloud-based AI / ML workflow and data application development framework

  bring-your-own-accountawsaiflowclouddata-scienceevent-basedlow-code-frameworkmachine-learningfeature-engineeringaws-glueaws-emrsagemaker-notebook-instancesagemaker-notebookaws-lambdaserverlesssparkpysparkscala-spark
  Language:Python 16

• jhole89 / aws-glue-sbt-quickstart

  Example of how to set SBT up for local development of AWS Glue Scripts

  sbtapache-sparkaws-gluequickstart
  Language:Scala 16

• aws-samples / amazon-rds-export-to-s3-automation

  This repository contains source code for the AWS Database Blog Post Reduce data archiving costs for compliance by automating RDS snapshot exports to Amazon S3

  amazon-athenaamazon-eventbridgeamazon-rdsamazon-s3amazon-snsaws-backupaws-cloudformationaws-glueaws-glue-crawleraws-kmsaws-lambda
  15

• jonrau1 / AWS-ComplianceMachineDontStop

  Proof of Value Terraform Scripts to utilize Amazon Web Services (AWS) Security, Identity & Compliance Services to Support your AWS Account Security Posture.

  automationawsaws-cognitoaws-configaws-glueaws-xraycloud-securitycompliancedevopsdevsecopsguarddutykinesis-firehoselambdamodulespythonremediationsecopssecurity-hubterraformwaf
  Language:HCL 15

• mincloud1501 / DevOps

  DevOps에 대한 개념 이해와 AWS 개발자 도구를 활용한 실습 및 연구

  codecommitcodebuildcodedeploycodepipelinecodestardevopsdevops-awscloud9awsaws-athenaaws-beanstalkaws-cloudtrailaws-glueaws-quicksightaws-codebuildaws-codestaraws-lambdaamazon-ec2devops-modeldevops-tools
  Language:Java 14

• vincentclaes / glue-devcontainer

  Glue VSCode devcontainer setup

  awsaws-gluegluevscode
  Language:Python 14
• Data-Engineering-Onboarding-Starter

  wednesday-solutions / Data-Engineering-Onboarding-Starter

  This repository contains a 10 step program to enter the world of Data Engineering

  awsaws-gluedatadata-engg-learningdata-engineeringdata-engineering-starterdata-templatedataenggdataengg-templateetlgluesparkworkflow
  Language:Python 14