Get Started with Databricks for Data Engineering

In this course, you will learn basic skills that will allow you to use the Databricks Lakehouse Platform to perform a simple data engineering workflow. You will be given a tour of the workspace, and you will be shown how to work with notebooks. You will create a basic data engineering workflow while you perform tasks like creating and using compute resources, working with repositories, and creating and managing basic workflow jobs. The course will also introduce you to Databricks SQL. Finally, you will see how data is stored, managed, governed, and secured within the lakehouse.

Languages Available: English | 日本語 | Português B

This course introduces you to evaluating and governing generative AI systems. First, you’ll explore the meaning behind and motivation for building evaluation and governance/security systems. Next, we’ll connect evaluation and governance systems to the Databricks Data Intelligence Platform. Third, we’ll teach you about a variety of evaluation techniques for specific components and types of applications. Finally, the course will conclude with an analysis of evaluating entire AI systems with respect to performance and cost.

The course is designed to provide you the practical experience in building advanced LLM applications using multi-stage reasoning LLM chains and agents. First, you’ll learn how to decompose a problem into its components and select the most suitable model for each step to enhance business use cases. Following this, we’ll show you how to construct a multi-stage reasoning chain utilizing LangChain and HuggingFace transformers. Finally, you’ll be introduced to agents and will design an autonomous agent using generative models on Databricks.