administrator 
Databricks: Spark Declarative Pipelines & Lakeflow Designer, Build end-to-end data pipelines in Databricks using SQL, Spark Declarative Pipelines, and Lakeflow Designer.
Description
Building data pipelines in Databricks used to mean a lot of notebook logic, Spark code, and manual orchestration.
But with Spark Declarative Pipelines and Lakeflow Designer, this changes.
In this course, you’ll learn how to build end-to-end data pipelines by defining what you want with SQL, while Databricks handles the execution, dependencies, and orchestration for you.
You’ll build a complete Bronze → Silver → Gold pipeline using a real e-commerce dataset. Starting from raw data, you’ll ingest, clean, transform, and aggregate it into analytics-ready tables.
Along the way, you’ll work with Delta Lake to ensure reliability and reproducibility, and use Unity Catalog to organize and govern your data.
Once the pipeline is built, you’ll schedule, run, and monitor it, turning it into a real, operational workflow.
Then, we will go one step further.
With Lakeflow Designer and Genie, you’ll learn how pipelines can be built visually, almost no-code, while still generating the same underlying logic.
As a bonus, we will also explore streaming pipelines using AWS Kinesis so you understand how the same declarative model works for near real-time data.
By the end of this course, you’ll understand a modern way of building data pipelines in Databricks, from raw data to production-ready workflows.
Who this course is for:
- Data Engineers who want to build structured pipelines in Databricks
- Analytics Engineers who want to move beyond notebook-based workflows
- Data Analysts who want to understand how modern pipelines are built
- Anyone learning Databricks and looking for a practical, hands-on project
- People who want to learn a simpler, more structured way of building pipelines
