This repository contains a set of pre-built dbt Load and Transform models for common SaaS applications to create an integrated ("conformed") data warehouse dimensional model. In its current incarnation it supports Google BigQuery as the target data warehouse and Stitch (and Segment warehouse destinations) as the data pipeline technology.
- To standardise how we source and model SaaS data sources in-general
- To make it simpler to run data quality tests than to not, by defining these tests in-advance
- To enable merging of customer, product, contact and other shared entity data with no single authoratitive source
- To pre-create derived analytics measures for individual and combinations of sources
- In-time, to create a means of selecting sources or subject areas ("modules") and have just those sources/modules loaded (and deployed for a customer)
- To do all of this in a way that embraces, rather than avoids, community additions to these sources and derived analytics models
See Design and Coding Approach for Implementation Details
- Hubspot CRM
- Harvest Timesheets
- Xero Accounting
- Stripe Payments
- Asana Projects
- Jira Projects
- Mailchimp Email Marketing
- Segment Events
- GCP Billing Exports
- Google Ads
- Facebook Ads
data sources on roadmap for Q2 2020 include:
- Mixpanel Events
- Salesforce CRM
- Bamboo HRMS
- Chargebee Payments
- Intercom Messaging
- Google Analytics (Free & 360)
- Finance (Invoices, Chart of Accounts, Currencies)
- CRM (Deals, Contacts, Companies)
- Projects (Timesheet Projects, Timesheet Tasks, Delivery Projects, Delivery Tasks, Timesheets, Users)
- Marketing (Email lists, Email sends, Email campaigns, Ad Campaigns, Ad Performance, Web Page Views, Web Sessions)
See Setting up a new Warehouse Environment for instructions on how to set-up your own dev environment, or a new client environment