mozilla / gcp-ingestion

Documentation and implementation of telemetry ingestion on Google Cloud Platform

https://mozilla.github.io/gcp-ingestion/

mozilla/gcp-ingestion Issues

Update Beam SDK version
Closed 6 months ago
Add test for flushing a queue with outstanding in-flight messages
Updated 8 months ago
Update google-cloud-pubsub in ingestion-edge
Updated 8 months ago1
Consider some message scrubbing before parsing JSON payloads
Updated a year ago
Branches, tags, and deployment strategy
Closed 2 years ago7
Set up dependabot for flex template base image
Updated 2 years ago
Update documentation for java build on various platforms
Updated 2 years ago
structured-decoder-loader latency event
Closed 2 years ago1
Link to deployment info in docs
Closed 2 years ago4
Documentation for flush manager
Updated 2 years ago1
test creation in gcp-ingestion github showing up in Jira DENG and tagged with component
Updated 2 years ago
Update CircleCI gh-pages deploy key
Updated 2 years ago
ingestion-beam/bin/generate-jwk-pair:40: DeprecationWarning: Call to deprecated function (or staticmethod) key_id
Updated 2 years ago
Refactor BQ table listings to a side input
Updated 3 years ago
Normalize namespace and docType within ParseUri
Updated 3 years ago1
Reuse bigquery payload format implementation between sink and beam
Closed 3 years ago1
Use Dataflow Flex Templates and remove use of ValueProvider
Closed 3 years ago5
ManagedChannelImpl was not shutdown properly
Updated 3 years ago1
Remove Redis interaction code
Closed 3 years ago
Design for supporting FxA event streaming to Amplitude
Closed 3 years ago3
Error message from worker: java.lang.NullPointerException com.mozilla.telemetry.decoder.rally.DecryptRallyPayloads$Fn.apply(DecryptRallyPayloads.java:188)
Closed 3 years ago3
Deploy generated javadocs alongside other documentation
Updated 3 years ago
Remove AET tests in ingestion-beam
Closed 3 years ago2
Update architecture diagram to remove Redis
Closed 3 years ago
Default to user agent for `normalized_os` if no `os` field is present
Updated 3 years ago
Ensure sample_id is always populated
Updated 3 years ago
Unused Field
Closed 3 years ago
structured decoder: Error message from worker: java.lang.NullPointerException
Closed 3 years ago
Publish packages to a repository accessibly by Maven
Closed 3 years ago1
`mvn spotless:apply` no longer works on JRE 8
Closed 4 years ago2
Unreleased Resource: Streams
Updated 4 years ago
Add BIG_QUERY_JOB_PROJECT or similar configuration option to ingestion-sink
Closed 4 years ago
ingestion-sink undeliverable message: java.lang.ClassCastException: null
Closed 4 years ago1
Investigate shutdown behavior for ingestion-sink
Closed 4 years ago1
Report total pipeline latency as custom metric in ingestion-sink
Closed 4 years ago1
ingestion-sink undeliverable message: java.lang.NullPointerException: null
Closed 4 years ago2
Handle conflicting "valid" keys better when writing to BigQuery
Updated 4 years ago
NullPointerException on empty metadataLocation when Pioneer is enabled
Closed 4 years ago
Running decoder locally fails with "com.maxmind.db.InvalidDatabaseException"
Closed 4 years ago
Guard against extra bytes when using BigQueryIO with direct reads
Closed 4 years ago
Decide on AET ecosystem_anon_id in header vs. payload
Closed 4 years ago9
Emit structured logs from ingestion-sink
Closed 4 years ago4
Database edition not found in `bin/download-geolite2`
Closed 4 years ago
Republish per docType avoid using `Partition`
Updated 4 years ago
Consider removing Aurora from normalized_channels
Updated 4 years ago3
Consider channel-specific main_v4 tables, filtering probes per channel
Updated 4 years ago4
beam live sink error: com.mozilla.telemetry.ingestion.core.util.BubbleUpException
Closed 4 years ago2
Filter countries with <15k population
Updated 4 years ago2
Remove schema alias support from ingestion-beam
Closed 4 years ago1
BQ sink template build failure
Closed 4 years ago1