Data Architecture Course

A hands-on course covering the architectural patterns, distributed systems concepts, and cloud technologies used to design and operate modern data platforms at scale.

What You'll Learn

Students progress from foundational processing paradigms to production-grade architecture patterns, grounding each topic in real technology stacks deployed on AWS.

Processing Architectures — Understand the trade-offs between batch and streaming processing. Study Lambda Architecture (batch + speed + serving layers) and Kappa Architecture (single streaming layer with replay), and when to choose one over the other.

Data Architecture Types — Survey the major patterns used in production: Event-Driven Architecture, Data Lake, Medallion (Bronze/Silver/Gold), Data Mesh, Data Vault, and Lakehouse. Each is evaluated on its strengths, failure modes, and fit for different organizational contexts.

Distributed Systems Foundations — Build the conceptual backbone needed to reason about any architecture: CAP Theorem, scalability strategies, disaster recovery (RTO/RPO), data ingestion patterns, storage tiers, networking, and governance.

Key Concepts — Develop fluency in the vocabulary of distributed data systems: ACID transactions, eventual consistency, idempotency, partitioning, sharding, schema evolution, replication, stream processing, and data contracts.

Technology Stack

The course is technology-grounded throughout, covering:

• Messaging & Streaming: Apache Kafka, Apache Flink
• Batch Processing: Apache Spark on AWS EMR
• Storage: AWS S3, Apache Iceberg, Delta Lake, Apache HudiData Architecture is a hands-on course for engineers who want to go beyond writing queries and start designing the systems that move, store, and serve data at scale. You'll learn when to stream and when to batch, how to pick the right architecture pattern for the job, and how to reason about trade-offs that matter in production — availability, latency, consistency, and fault tolerance. By the end, you'll have designed Lambda and Kappa architectures from scratch and built a real pipeline for an on-demand mobility platform that hits production-grade SLAs, validated by an automated grader.

• Serving: AWS Redshift, AWS DynamoDB, Snowflake
• Orchestration: Apache Airflow, Dagster
• Governance & Cataloging: AWS Glue Data Catalog, OpenMetadata
• Infrastructure: Terraform, AWS ECS/EKS
• Observability: Prometheus, Grafana

Hands-On Work

Exercises — Students design and evaluate Lambda and Kappa architectures against rubrics covering design rationale, technology choices, and CAP classification.

Architecture Lab — A capstone lab built around an on-demand mobility platform. Students design a pipeline that meets production SLA targets: 99.9% availability, ≤3-second end-to-end latency, ≥99% field completeness, and ≤0.1% duplicate events. The pipeline is validated by an automated Dagster grader.

Prerequisites

Familiarity with SQL and basic cloud concepts is assumed. No prior distributed systems experience required.