Get Started with Databricks for Data Engineering

In this course, you will learn how to leverage Databricks' data sharing capabilities to enable secure, efficient collaboration across your organization and with external partners. The course covers the fundamentals of modern data sharing, exploring the business value it delivers and examining Databricks' three core solutions: Delta Sharing, Marketplace, and Clean Rooms. You'll understand the architecture of Delta Sharing, including both Databricks-to-Databricks (D2D) and open (D2O) sharing models, and learn how to share diverse asset types such as tables, AI models, and notebooks. The course further explores advanced replication strategies, cross-cloud and cross-region optimization, and implementation of granular access controls using dynamic views and partition filtering. You'll also master security and governance practices, including token management, fine-grained access control, cost optimization, and auditing strategies. By the end, you'll have the skills to design secure Delta Sharing solutions for internal and external use cases, optimize multi-party collaboration while managing costs, and choose the right Databricks sharing solution for your specific business needs.

In this course, you will develop an in-depth understanding of how to design, implement, and govern scalable machine learning systems that operate effectively at enterprise scale. The curriculum is organized into three experiential modules: developing distributed ML workflows with frameworks such as Apache SparkML and Ray, transitioning local ML development to distributed compute using tools like Pandas on Spark, and operationalizing and governing production models with Databricks’ MLOps ecosystem.

Through hands-on projects, you will construct end-to-end distributed ML pipelines using the SparkML workflow, applying Transformers, Estimators, and the fit/transform paradigm for both classification and regression tasks. You will version, compare, and manage experiments using MLflow 3.0 to ensure reproducibility and governance, capturing lineage between data, features, and model artifacts. Additionally, you will apply scalable Hyperparameter Optimization frameworks to improve model performance at scale.

The course concludes by demonstrating complete lifecycle management, from experimentation to production deployment, using Unity Catalog and Model Serving. You will learn to operationalize trained models, monitor their performance, and implement strong governance over models, features, and Delta assets within the Databricks environment.

このコースでは、ハンズオンデモとキャップストーンラボを使用して、Unity Catalog および Databricks でのきめ細かなアクセス制御について説明します。 テーブルの種類、カタログとスキーマの構成、グループベースのアクセス管理、およびアクセス制御の移行戦略について学習します。 このコースには、行レベルのセキュリティと列マスキングによるきめ細かなアクセス制御の適用、属性ベースのアクセス制御、制御の組み合わせ、制御の移行、および包括的なガバナンス実装のためのラボに関するデモが含まれています。