All projects B2B SaaS · publicly traded

Snowflake Analytics for a Public SaaS Company

A Gartner-leader SaaS product company

A Snowflake warehouse serving internal BI, self-service, customer-facing insights and ML features.

Snowflake dbt Apache Kafka Fivetran Looker Tableau Apache Airflow Cube.js Terraform Amazon S3

The team

  • Data Platform team, ~8 engineers, managing the Kafka stream
  • Data Pipeline team, ~6 engineers, managing ingestion into Snowflake
  • Data Warehouse team, ~6 engineers, managing core DW models
  • Separate ML and BI teams
  • 1 Senior Manager and 1 VP of Data

Use cases

  • Data warehouse and BI reporting in Tableau
  • Self-service reporting in Looker and Snowsight
  • Customer-facing data insights
  • ML features for the product
  • Technical sales team dashboards

Architecture

Architecture with MongoDB, Postgres and product logs through Kafka to an S3 landing bucket, into Snowflake external, staging, core and datamart tables, out to Tableau and Looker
Kafka to S3 to Snowflake external tables, then staging, core and team datamarts serving Tableau, Looker and ML products.

What was built

  • Snowflake as the core data warehouse
  • dbt core for all Snowflake transformations
  • Kafka Connect writing MongoDB and application logs to S3, read through Snowflake external tables
  • Fivetran for Salesforce, Marketo, Google Sheets and similar sources
  • Looker for self-service BI, Tableau for curated BI
  • Cube.js powering customer-facing visualisations
  • Airflow for orchestration, with dbt running on ECS
  • Terraform for infrastructure and Snowflake itself, with Git and CI/CD

What changed

  • One warehouse served four distinct audiences: internal BI, self-service analysts, the product's own customers, and ML.
  • Splitting platform, pipeline and warehouse into separate teams scaled delivery — at the cost of fewer people holding the end-to-end picture.
  • Snowflake and its access model were managed in Terraform, so environment changes went through review like any other code.
Related service

Snowflake Consulting

Snowflake implementation, dbt modeling, and credit cost optimization.

Explore service

Let's talk about your data platform

Tell us what you are working on. We reply within one business day, and we will tell you plainly if we are not the right fit.

Start a conversation Email us

Other projects

E-commerce marketplace · 400 employees

Oracle Data Warehouse to AWS Redshift

AbeBooks (an Amazon subsidiary)

Migrating a marketplace off Oracle, PL/SQL and Crystal Reports onto Redshift and Tableau.

Amazon Redshift Matillion ETL Tableau AWS EMR +4 more
Read the project Consumer technology · internal data products

Product Analytics for a Voice Assistant

Amazon Alexa — Natural Language Understanding

Replacing manual CSV extracts with an automated Redshift and Tableau product analytics stack.

Amazon Redshift Redshift Spectrum Tableau Amazon SageMaker +1 more
Read the project E-commerce · machine learning

Feature Engineering for Machine Learning

Amazon Retail

Building ML feature pipelines from a core data lake — with no warehouse, no data model and no dashboards.

Apache Spark Scala AWS EMR Amazon S3 +3 more
Read the project