Agentic Systems and Evaluation with Databricks AI

Databricks lets you build and deploy your AI Agent System for any use case, leveraging foundation LLMs, PDF extraction, Vector Search, and Mosaic AI Agent Evaluation.

In this demo, you'll learn how to:

• Build and deploy your first tools, and save them as a Unity Catalog function

• Create your first agent, leveraging these tools through LangChain

• Evaluate your agent and build an evaluation loop to ensure new versions perform better on your dataset

• Leverage MLflow 3.0 tracing, feedback, and scorers, including evaluation dataset generation

• Package and deploy your chatbot as a Databricks application

• Scan and extract information using Databricks' built-in document_ai_parse function

• Monitor your live, production agent behavior with Databricks Monitoring

To run the demo, get a free Databricks workspace and execute the following two commands in a Python notebook: