Course Overview
The Microsoft Azure Databricks Data Engineer Associate (DP-750) course is designed for data engineers and analytics professionals who build scalable data engineering solutions using Azure Databricks and Apache Spark on Microsoft Azure. This instructor-led training provides comprehensive coverage of designing, implementing, securing, and optimizing modern data pipelines using the Databricks Lakehouse Platform.
Participants will learn how to configure Azure Databricks workspaces, manage compute resources, develop notebooks, ingest data from multiple sources, transform large datasets, and build reliable batch and streaming pipelines. The course covers core data engineering technologies including Delta Lake, Auto Loader, Structured Streaming, Unity Catalog, SQL Warehouses, and Databricks Workflows to support enterprise-scale analytics solutions.
Learners will gain practical experience implementing the Medallion Architecture (Bronze, Silver, and Gold layers), managing Delta Lake tables, optimizing Spark workloads, implementing enterprise data governance, and securing data assets using Microsoft Entra ID integration, Unity Catalog, Azure Key Vault, and role-based access control. The course also introduces Lakeflow Jobs for workflow orchestration and explores performance optimization techniques using Photon, Adaptive Query Execution (AQE), partitioning, and caching.
Extensive hands-on labs and real-world implementation scenarios enable participants to build end-to-end data engineering solutions using Azure Databricks. By the end of the course, learners will be equipped with the practical skills required to design secure, scalable, and governed Lakehouse solutions and prepare confidently for the Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750) certification.
Course Objective
Upon successful completion of this course, participants will be able to:
- Understand the architecture and core components of Azure Databricks and the Databricks Lakehouse Platform.
- Configure Azure Databricks workspaces, compute resources, SQL Warehouses, and development environments.
- Develop collaborative notebooks using Python, SQL, Spark SQL, and Apache Spark.
- Ingest structured, semi-structured, and streaming data using Azure Data Lake Storage, Auto Loader, and Apache Spark.
- Build scalable ETL and ELT pipelines using Delta Lake and Medallion Architecture principles.
- Implement Delta Lake features including ACID transactions, schema evolution, Change Data Feed (CDF), time travel, and Liquid Clustering.
- Build real-time streaming pipelines using Structured Streaming, checkpoints, and event-time processing.
- Implement enterprise data governance using Unity Catalog, storage credentials, external locations, data lineage, row filters, and column masking.
- Secure Azure Databricks environments using Microsoft Entra ID, Azure Key Vault, cluster policies, and role-based access control.
- Create and automate production data engineering workflows using Databricks Jobs and Lakeflow Jobs.
- Optimize Apache Spark workloads using Photon, Adaptive Query Execution (AQE), partitioning, caching, and query optimization techniques.
- Monitor and troubleshoot Azure Databricks workloads using Spark UI, Azure Monitor, query history, and cluster diagnostics.
- Integrate Azure Databricks with Azure Data Factory, Azure Event Hubs, Azure Monitor, Azure Key Vault, GitHub, and Azure DevOps.
- Design secure, scalable, and governed enterprise Lakehouse solutions.
- Prepare for Exam DP-750 and earn the Microsoft Certified: Azure Databricks Data Engineer Associate certification.
Pre-requisites
Participants should have:
- Basic understanding of data engineering concepts and ETL processes.
- Working knowledge of relational databases and SQL.
- Familiarity with Microsoft Azure fundamentals and cloud computing concepts.
- Basic programming knowledge in Python and SQL.
- Understanding of Azure Storage services such as Azure Data Lake Storage Gen2.
- Familiarity with data transformation concepts and distributed data processing is recommended.
- Prior exposure to Apache Spark or big data technologies is beneficial but not mandatory.
- Experience with Azure Data Factory or modern analytics platforms is advantageous.
- Knowledge equivalent to Microsoft Azure Fundamentals (AZ-900) is recommended.
Course Curriculum
