All projects B2B SaaS

Analytics for a Pre-IPO SaaS Product

A Canadian SaaS product company (pre-IPO)

An open-source data stack on AWS Athena, carrying finance reporting through an IPO.

AWS Athena dbt Meltano Looker Metabase AWS Glue Apache Spark Prefect Terraform Amazon Redshift Snowplow

The team

  • 6 Data Engineers
  • 6 Analytics Engineers (dbt, SQL, Looker)
  • 2 Product, 2 Marketing and 4 Financial analysts
  • Managers for DE, AE, Product, Marketing and Finance
  • 1 Senior Manager Analytics, 1 Director of Data Engineering, 1 VP Data

Starting state

  • An open-source stack carrying real technical debt
  • A dbt adapter for Athena
  • A data solution originally built by DevOps

Use cases

  • Financial, marketing and product reporting
  • BI dashboards
  • Finance reporting for the IPO
  • Product telemetry project with Snowplow

Architecture

Data platform architecture with sources, ingestion, raw and clean storage, analysis and BI, reverse ETL, orchestration and observability
The full open-source platform: ingestion, raw and clean storage on S3, Athena for analysis, plus reverse ETL and observability.

What was built

  • Meltano for extraction
  • dbt core targeting Athena, and later Redshift
  • Athena as the SQL engine over a Parquet data lake
  • Looker as the BI layer, Metabase for ad-hoc SQL
  • Glue and Spark for product and clickstream log processing
  • Prefect for orchestration, running on ECS alongside dbt
  • Terraform for data engineering infrastructure, with Git and CI/CD

What changed

  • Financial reporting held up to the scrutiny that comes with an IPO process.
  • Infrastructure moved into Terraform and code review, away from a DevOps-built solution nobody on the data team owned.
  • The team moved to Redshift for the warehouse to get past dbt-on-Athena constraints — a reminder that open source shifts cost rather than removing it.
Related service

Cloud Data Platforms

Architect and build cloud data platforms that scale to petabytes.

Explore service

Let's talk about your data platform

Tell us what you are working on. We reply within one business day, and we will tell you plainly if we are not the right fit.

Start a conversation Email us

Other projects

E-commerce marketplace · 400 employees

Oracle Data Warehouse to AWS Redshift

AbeBooks (an Amazon subsidiary)

Migrating a marketplace off Oracle, PL/SQL and Crystal Reports onto Redshift and Tableau.

Amazon Redshift Matillion ETL Tableau AWS EMR +4 more
Read the project Consumer technology · internal data products

Product Analytics for a Voice Assistant

Amazon Alexa — Natural Language Understanding

Replacing manual CSV extracts with an automated Redshift and Tableau product analytics stack.

Amazon Redshift Redshift Spectrum Tableau Amazon SageMaker +1 more
Read the project E-commerce · machine learning

Feature Engineering for Machine Learning

Amazon Retail

Building ML feature pipelines from a core data lake — with no warehouse, no data model and no dashboards.

Apache Spark Scala AWS EMR Amazon S3 +3 more
Read the project