bigspark's repositories
genai-presidio
Repository for PII Anonymizer code package and sample FastAPI API to use it to talk to LLM
aws-sso-sync
sso-sync tool to help with the SCIM setup for bigspark.
test_glue_
To test glue job
test_glue
To test glue job
ai-hackathon
General Purpose repo for NW AI Hackathon 2023
streamsets_json_schema_validator_processor
A streamsets dc sample processor for validation records with a specified JSON schema
incubator-livy
Mirror of Apache livy (Incubating)
snowpark-scala-data-profiling
Data profiling example using Snowflake sample datasets and Scala
sparkMeasure
This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.
jvm-profiler
JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter
emr-uber-profiler-notebooks
emr-uber-profiler-notebooks
deequ
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
tutorials
StreamSets Tutorials
kafka-local
Basic single broker Kafka cluster - docker compose using confluent image
kafdrop
Kafka Web UI
CMAK
CMAK is a tool for managing Apache Kafka clusters