Articles
Browse by topic and level, from your first data flow to pipeline recovery.
Filters and display
9 articles · All topics · All levels · Newest · Page 1/2
Designing Idempotent and Reliable Pipelines
Use a business key and source revision to make a destination update safe to replay.
Understanding Event Streaming with Kafka
Reason about partition order, consumer groups, offsets, and duplicate delivery using a small event log.
Tested Data Transformations with dbt
Connect a model with ref, test its business key, and review incremental correctness separately.
Scheduling Pipelines with Apache Airflow
Define task dependencies and bounded retries while keeping batch publication safe to repeat.
Understanding Dimensional Modeling and Star Schemas
Choose a sales fact grain before joining descriptive dimensions and aggregating revenue.
Building an ETL Pipeline with Python
Read a small CSV, reject invalid amounts explicitly, and write a repeatable JSON Lines output.
SQL for Cleaning Transaction Data
Normalize missing identifiers, choose a deterministic duplicate winner, and reconcile rejected transactions.
Understanding Data Engineering and Its Workflow
Follow sample transactions from their source to a table ready for analysis.