Try Dagster Multi-assets to handle daily/weekly/monthly aggregations

[Raymond Cheng]

What is it?

https://docs.dagster.io/concepts/assets/multi-assets

[Raymond Cheng]

Let's remember to version any marts generated by this as _v1

[Raymond Cheng]

Here's the current thought on what we want to do here:

• Right now we aggregate and count different event-types on different bucket sizes.
• In reality, we want to have different metric_types. For example, instead of having star event aggregations, we have a metric called star_count_1_day or star_count_1_week etc. This is in effect the star count from the last 1 day/week/month
• We want to be able to have time-series data for any metric. So the table schema would look like timestamp, project_id, metric_type, amount.
• We can then use Dagster multi-assets to auto-generate multiple related metrics.

[Raymond Cheng]

This isn't a requirement for RF4 so going to push this till after RF4.