#data-engineering

Design a Large-Scale Data Pipeline (ETL / Batch + Streaming)

Move and transform petabytes from sources into a warehouse/lake for analytics. DAG orchestration, Spark shuffles, lake vs warehouse, and idempotent, replayable jobs.