snowpipe

There are 0 repository under snowpipe topic.

• sfguide-twitter-auto-ingest

  Snowflake-Labs / sfguide-twitter-auto-ingest

  Learn how to auto-ingest streaming data into Snowflake using Snowpipe.

  snowpipesnowflaketwitterdemo
  Language:Python 23

• enfazz / aws_snowflake_pbi

  AWS S3, Snowflake (Snowpipe, Snowpark) and PowerBi.

  awspowerbis3snowsnowparksnowpipe
  Language:PLpgSQL 5

• pedrogfleming / Snowflake-Scripts

  SQL Scripts related to my learning on the Snowflake data cloud provider

  clouddatadata-analysissnowflakesqljson-parsingunstructured-datadatawarehouseawss3failsafetimetraveletlsnowpipe
  4

• phdata / snowpipe-cli

  CLI script for interacting with Snowpipe REST APIs

  snowpipesnowflakecli
  Language:Python 4

• Snowflake-Labs / simstreamer

  Generating Streaming Data for Snowpipe Streaming

  snowpipestreaming
  Language:Java 4

• arunp77 / Database-datapipeline-ETL

  SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline

  apache-airflowapache-nifiawsaws-cloudformationaws-ec2aws-lambdaaws-s3data-pipelinedatawarehouseetletl-pipelinesnowflakesnowflake-schemasnowpipe
  Language:Jupyter Notebook 3

• arunp77 / SQL

  SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline

  airflow-dagsaws-ec2aws-s3data-pipelinedatabase-managementdatawarehouseelt-pipelineetl-pipelinemysql-databasesnowflakesqlapache-airflowapache-nifisnowflake-schemasnowflakessnowpipe
  3

• Abhishek010397 / Azure-Snowflake

  azuresnowflakeazureblobstoragesnowflake-stagessnowpipeazure-queueazure-event-gridmergetask
  2

• alejlatorre / kafka-spark-smartcity-realtime-pipeline

  Realtime data pipeline using Kafka + Spark + AWS S3 (Terraform) + Snowflake

  awskafkasparkterraformsnowflakesnowpipe
  Language:Python 2

• Davidelvis / Snowflake-Azure-Project-to-build-real-time-Twitter-feed-dashboard

  apisnowflakesnowpipeazureazure-storagepythontwitter-api
  Language:Python 1

• dominicho97 / Snowpipe

  Automatically move data from AWS S3 to Snowflake datawarehouse continuously with snowpipe

  aws-s3snowflakesnowpipe
  1

• miaaaalu / Using-Streams-and-Tasks-In-Snowflake

  snowflakesnowpipesnowflake-streamaws-s3
  1

• psy-pri / aws-realtime-tweets-streaming

  ❄️ Stream real-time data into Snowflake with Amazon Kinesis Firehose

  awsaws-kinesis-firehoseaws-s3beginner-friendlyboto3boto3-scriptdatapipesnowflakesnowpipetweepytwitter-api
  Language:Python 1

• SirNicholas1st / Snowflake-Datapipe

  The goal is to build an example of a simple data collection pipeline which collects data from multiple customers and uploads the data to Snowflake could look like. The OpenMeteo Api acts as the "customer system" in this case.

  awsaws-lambdasnowflakesnowpipe
  Language:Python 1

• WilsonH918 / EnergyStocks_HistoricalPrice_DataPipeline

  awsaws-lambdapipelinepythonec2-instancesnowflakesnowpipeaws-s3
  Language:Python 1

• amoghkori / retail_data_analysis

  Retail data analysis pipeline utilizing AWS S3, Snowflake, Python, SQL, and Tableau. It demonstrates data transformation and setup in Jupyter Notebook, integrates real-time retail insights via an automated Tableau dashboard with Snowflake, and employs a CRON job in Jupyter Lab connected to Amazon SQS for consistent data updates.

  aws-s3awssqscronjobjupyter-notebookplpgsqlsnowflakesnowpipesqltableau
  Language:Jupyter Notebook 0

• miaaaalu / Cloud-ETL-Data-Integration-from-AWS-to-Snowflake

  aws-lambdaserverless-frameworksnowflakesnowpipestar-schema-tables
  Language:Jupyter Notebook 0

• nagavenkateshgavini / spotify_data_pipeline

  Spotify ETL pipeline on AWS

  awsaws-lambdaaws-s3awsiametlpythonsnowflakesnowpipe
  Language:Python 0

• ravishankar324 / Real-Time-Insurance-claims-Data-ETL-Pipeline

  Real-time ETL (Extract, Transform, Load) data pipeline to process insurance claims data with Snowflake, Apache Airflow, AWS S3, EC2, python pandas and creating a real time data visualization dashboard using Tableau.

  apacheairflowapiaws-ec2aws-s3kagglesnowflakesnowpipetableausnowflakestreamssnowflaketasks
  Language:Python 0

• sarutlaa / AWS-Snowflake-Music-Insights-Pipeline

  Building a Seamless Data Pipeline with AWS and Snowpipe

  aws-lambdaaws-s3data-warehousepandaspythons3-event-notificationsnowflakesnowpipesql
  Language:Jupyter Notebook 0

• Srilekha-1106 / Snowflake-Project

  Implemented Snowflake project on AWS for efficient data storage and transformation. Utilized JSON-to-CSV conversion, Snowpipe for real-time ingestion, and reader accounts for secure data access. Employed streams, tasks, and materialized views for data synchronization and optimization. Implemented masking policies for enhanced data security.

  awscsvdata-transformationdatabaseetljson-parserschemasnowflakesnowpipesqlstorage-integrationwarehouse
  0

• areesha-s / Redfin-Data-Pipeline-with-AWS-Apache-Airflow-Snowflake-and-PowerBI

  This project showcases a scalable ETL pipeline that automates the extraction, transformation, and storage of Redfin housing market data using AWS, Apache Airflow, and Snowflake, with Power BI for data visualization. The pipeline is configured to run monthly, ensuring your data remains up-to-date.

  awsetl-pipelinepowerbipythonsnowflakesnowpipe
  Language:Jupyter Notebook

• dogukannulu / snowpipe-aws-stream-processing

  Get the streaming data from the S3 bucket with SQS queue. Load into Snowflake with Snowpipe and modify the data with Snowflake task

  airflowawsdockers3snowflakesnowpipesqs
  Language:Python

• FelopateerEkladious / kinesis-snowflake-pipeline

  Analyze real-time global market data using AWS Kinesis and Snowflake. We utilize CSV datasets extracted via API calls, stream them through Kinesis Firehose, and transform them with Snowflake. Our agile workflow ensures efficiency, providing a one stop comprehensive solution for real-time data insights.

  apprunnerawsetl-pipelinekinesiskinesis-firehosekinesis-streamlambdapowerbipythons3snowflakesnowparksnowpipesqlreal-time
  Language:Jupyter Notebook

• gakas14 / Event-driven-architecture-with-S3-Lambda-snwoflake

  ETL with AWS Lambda and Snowflake with automation using Snowpipe.

  iamroleslambda-functionslambda-layerspython3s3snowflakesnowpipe

• kabeera1007 / Cricket-ETL-Pipeline

  End-to-End ETL Pipeline for Live Cricket Streaming Data Using AWS (Lambda, Glue, Step Functions, S3), Snowflake, and Power BI for Visualization

  awsdatabricksgluelambda-functionspowerbipythons3-bucketsnowflakesnowpipesparksql
  Language:Jupyter Notebook

• svc98 / AWS_Snowflake_Migration

  Designed a cloud-to-cloud migration pipeline through migrating data with event-driven architecture and automated data loads. Implemented data quality checks at each stage, ensuring data consistency and reliability.

  awsdynamodbeventbridgelambdas3snowflakesnowpipesqs
  Language:Python