Buzzebees, Data platform
Customer Data Platform
Ingestion, history, identity resolution and publication: one profile per customer, built for analytics and journeys.
How it flows
- 1Files, databases, feeds
- 2S3 landing
- 3Typed history
- 4Quality
- 5Identity matching
- 6ClickHouse profiles
- 7Journey audiences
The problem
Customer, purchase and point data arrived from files, operational databases, historical snapshots and daily rolling feeds. The same person showed up many times under different keys, and no team could trust a single customer count.
My role
Designed and built the platform end to end: infrastructure as code, ingestion workers, typed history, quality rules, identity matching, publication into ClickHouse and the control API used by Journey Builder.
How it works
Every source lands in S3 first, then moves into typed PostgreSQL history split into raw, quality, identity and control schemas. Matching runs as its own service and writes through an outbox, so publication to ClickHouse is replayable. Historical snapshots go through a separate standardization path with receipts, so a restored source is never mistaken for a finished one.
Outcome
Profile, purchase and point views that analytics and marketing automation read from directly, with immutable audience snapshots so a journey always acts on the exact list that was approved.
What I learned
- Identity is a product decision before it is an algorithm. Write the matching rules down with the business first.
- Make every publication replayable. You will need to rebuild a profile table sooner than you think.
- Track completion explicitly. "The data is restored" and "the data is published" are different states.
Built with
- AWS CDK
- S3
- AWS Glue
- Fargate
- Step Functions
- PostgreSQL
- ClickHouse
- Elasticsearch
- Kafka
- Temporal