Skip to content
← Back to the foundry
CASE 04Case study · XL Dynamics

Validating and reconciling 20M+ records

Validation and reconciliation over 20M+ records.

  1. Source records
  2. Validation
  3. Reconciliation
  4. Trusted output
20M+
records validated and reconciled

Context

Associate Data Engineer at XL Dynamics, October 2020 to August 2024.

What I did

Built validation and reconciliation over 20M+ records.

Result

In the machine

The real pipeline in /pipeline reconciles every run the same way: bronze rows must equal silver plus quarantine plus duplicates, and gold totals must equal silver.