aws-glue

There are 9 repositories under aws-glue topic.

• aws-sdk-pandas

  aws / aws-sdk-pandas

  pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

  amazon-athenaamazon-sagemaker-notebookapache-arrowapache-parquetathenaawsaws-glueaws-lambdadata-engineeringdata-scienceemretlglue-cataloglambdamodinmysqlpandaspythonrayredshift
  Language:Python 3935
• awesome-aws-workshops

  dgomesbr / awesome-aws-workshops

  (Unofficial) curated list of awesome workshops found around in the internet. As we all have been there, finding that workshop that you have just attended shouldn't be hard. The idea is to provide an easy central repository, in a collaborative way.

  amazon-eks-workshopamazon-sagemakeramazon-sagemaker-workshopamazon-textractanalyticsapplication-modernizationawsaws-glueaws-iamaws-iotaws-workshopsimmersion-dayiot-awssagemaker-workshopserverlesssql-server
  Language:HTML 411

• tokern / piicatcher

  Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

  aws-athenaaws-gluepiidatabasedata-catalogphidatacatalogaws-redshiftsnowflakepython
  Language:Python 281
• dataall

  data-dot-all / dataall

  A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

  awsaws-glueaws-lake-formationaws-s3datadata-scienceetl-frameworklakeformationlakehouseredshift
  Language:Python 235

• aws-samples / data-lake-as-code

  Data Lake as Code, featuring ChEMBL and OpenTargets

  awsaws-cdkaws-cdk-constructsaws-glueaws-lake-formation
  Language:TypeScript 166

• awslabs / athena-glue-service-logs

  Glue scripts for converting AWS Service Logs for use in Athena

  alb-logsathenaaws-gluecloudfront-logscloudtrail-logselb-logsglue-jobglue-scriptss3-log-parservpc-flow-logs
  Language:Python 142
• cloud-experiments

  aws-samples / cloud-experiments

  Open innovation with 60 minute cloud experiments on AWS

  aws-gluedata-scienceamazon-athenaamazon-sagemakeramazon-s3machine-learningamazon-comprehendamazon-rekognitionnotebooksaws-cloud
  Language:Jupyter Notebook 88

• aws-samples / amazon-deequ-glue

  Automated data quality suggestions and analysis with Deequ on AWS Glue

  awsaws-gluedata-qualitydeequ
  Language:Scala 83
• streamlit-application-deployment-on-aws

  aws-samples / streamlit-application-deployment-on-aws

  Streamlit EDA Dashboard Powered by AWS Cloud

  streamlit-dashboardawsaws-cloudformationaws-glueaws-athenaaws-cognitoaws-sagemaker
  Language:Python 80

• aws-samples / analyzing-reddit-sentiment-with-aws

  Learn how to use Kinesis Firehose, AWS Glue, S3, and Amazon Athena by streaming and analyzing reddit comments in realtime. 100-200 level tutorial.

  kinesis-firehosedelivery-streamaws-gluedata-streamamazon-athenadata-lakeredditsentiment-analysissentiment-classificationreal-timeself-learningtutorials
  Language:Python 44

• aws-samples / bring-your-own-data-labs

  Bring your own data Labs: Build a serverless data pipeline based on your own data

  analyticsawslabshands-on-labaws-glueaws-s3aws-quicksight
  Language:HTML 42

• aws-samples / aws-glue-jobs-unit-testing

  Demo code to illustrate the execution of PyTest unit test cases for AWS Glue jobs in AWS CodePipeline using AWS CodeBuild projects

  automated-testingawsaws-gluepytest
  Language:Python 39

• Ditectrev / Amazon-Web-Services-Certified-AWS-Certified-Machine-Learning-MLS-C01-Practice-Tests-Exams-Question

  ⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

  mls-c01awsmachine-learningapache-sparkamazon-athenaamazon-ec2amazon-emramazon-sagemakeraws-glueaws-certifiedaws-certified-machine-learningamazon-cloudwatchamazon-transcribeaws-batchamazon-comprehendaws-lambdaamazon-s3amazon-textractlinear-regressionneural-network
  39

• DisasterAWARE / aws-glue-schema-registry-python

  Use the AWS Glue Schema Registry in Python projects.

  avroawsaws-gluekafkapythonschema-registry
  Language:Python 32

• 1oglop1 / aws-glue-monorepo-style

  Example of AWS Glue Jobs and workflow deployment with terraform in monorepo style. Code here supports the miniseries of articles about AWS Glue and python.

  pythonawsaws-glueserverlessdatascienceterraform
  Language:Python 30

• awslabs / amazon-athena-cross-account-catalog

  🌉 Reference implementation for granting cross-account AWS Glue Data Catalog access from Amazon Athena

  amazon-athenaaws-glue
  Language:Python 29

• GorillaStack / athena-cloudtrail-partitioner

  Automate the daily partitioning of your CloudTrail bucket in Athena

  awsathenaaws-athenacloudtrail-logscloudtrailpartitioninggorillastackglueaws-glue
  Language:JavaScript 28

• cloudposse / terraform-aws-glue

  Terraform modules for provisioning and managing AWS Glue resources

  awsaws-glueetletl-jobglueworkflow
  Language:HCL 27

• vincentclaes / serverless_data_pipeline_example

  Build and Deploy A Serverless Data Pipeline on AWS

  awsaws-glueaws-lambdaaws-s3pythonserverless-framework
  Language:Python 27

• aws-samples / transactional-datalake-using-apache-iceberg-on-aws-glue

  Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS

  apache-icebergaws-glueaws-dmsaws-athenaapache-spark
  Language:Python 25

• tokern / lakecli

  A CLI to manage and monitor permissions in AWS Lake Formation

  awsaws-glueaws-lake-formationpermissionssql
  Language:Python 25

• mikaelahonen-solita / aws-glue-tutorial

  AWS Glue tutorial for data developers.

  sparkpysparkaws-gluetutorial
  Language:Python 23

• moritzkoerber / covid-19-data-engineering-pipeline

  A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.

  apache-airflowapache-sparkapiawsaws-cdkaws-cloudformationaws-ecraws-glueaws-lambdaaws-redshiftaws-s3dockergreat-expectationspysparkspark
  Language:Python 23

• chgasparoto / terraform-aws-glue

  Terraform module which creates Glue resources on AWS

  awsaws-glueterraformterraform-modules
  Language:HCL 22

• webysther / aws-glue-docker

  🐋 Docker image for AWS Glue Spark/Python

  apache-arrowawsaws-cliaws-glueaws-glue-dockercdkdata-engineeringdevelopmentdockerdocker-imagedockerfileetlglue-catalogglue-pysparkpandaspytestpythonpython-poetrysamspark
  Language:Dockerfile 22

• aws-samples / aws-glue-streaming-etl-with-apache-iceberg

  Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3

  aws-glueapache-icebergaws-athenaapache-sparkaws-glue-streaming
  Language:Python 19

• aws-samples / monitoring-apache-iceberg-table-metadata-layer

  Sample code to collect Apache Iceberg metrics for table monitoring

  apache-icebergawsaws-cloudwatchaws-glueaws-lambdadata-qualitymonitoringsam-cliapache-sparkpyiceberg
  Language:Python 19

• aws-samples / aws-glue-crawler-utilities

  This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS CDK applications.

  aws-glueaws-glue-crawler
  Language:Python 18

• andreichiro / data_engineer_end2end

  End-to-end data engineer project

  ansible-playbookapi-restaws-api-gatewayaws-ecs-fargateaws-glueaws-lambdaaws-s3databricksdbt-clouddockerpandaspysparkpythonsqlterraformvitrinedev
  Language:HTML 17

• amzn / rheoceros

  Cloud-based AI / ML workflow and data application development framework

  aiawsaws-emraws-glueaws-lambdabring-your-own-accountclouddata-scienceevent-basedfeature-engineeringflowlow-code-frameworkmachine-learningpysparksagemaker-notebooksagemaker-notebook-instancescala-sparkserverlessspark
  Language:Python 16

• jhole89 / aws-glue-sbt-quickstart

  Example of how to set SBT up for local development of AWS Glue Scripts

  apache-sparkaws-gluequickstartsbt
  Language:Scala 16

• aws-samples / amazon-rds-export-to-s3-automation

  This repository contains source code for the AWS Database Blog Post Reduce data archiving costs for compliance by automating RDS snapshot exports to Amazon S3

  amazon-athenaamazon-eventbridgeamazon-rdsamazon-s3amazon-snsaws-backupaws-cloudformationaws-glueaws-glue-crawleraws-kmsaws-lambda
  15

• jonrau1 / AWS-ComplianceMachineDontStop

  Proof of Value Terraform Scripts to utilize Amazon Web Services (AWS) Security, Identity & Compliance Services to Support your AWS Account Security Posture.

  terraformawsguarddutymoduleslambdapythonremediationautomationkinesis-firehosewafsecurity-hubaws-glueaws-configdevopscloud-securitycomplianceaws-cognitoaws-xraydevsecopssecops
  Language:HCL 15

• mincloud1501 / DevOps

  DevOps에 대한 개념 이해와 AWS 개발자 도구를 활용한 실습 및 연구

  codecommitcodebuildcodedeploycodepipelinecodestardevopsdevops-awscloud9awsaws-athenaaws-beanstalkaws-cloudtrailaws-glueaws-quicksightaws-codebuildaws-codestaraws-lambdaamazon-ec2devops-modeldevops-tools
  Language:Java 14

• vincentclaes / glue-devcontainer

  Glue VSCode devcontainer setup

  awsaws-gluegluevscode
  Language:Python 14
• Data-Engineering-Onboarding-Starter

  wednesday-solutions / Data-Engineering-Onboarding-Starter

  This repository contains a 10 step program to enter the world of Data Engineering

  awsaws-gluedatadata-engineeringetlgluesparkworkflowdata-templatedataenggdata-engg-learningdata-engineering-starterdataengg-template
  Language:Python 13