How to Solve the Context Gap: Engineering Reliable AI Agents

The conversation is grounded in hands-on patterns and the capabilities of the Databricks Platform, including Agent Bricks, Genie, Lakebase, MLflow, and Unity AI Gateway. Longer context windows do not automatically yield better results, so treating context as a first-class artifact is what separates demos from durable systems.

Why longer context alone degrades reliability

Benchmarks such as needle-in-a-haystack show models can pluck a single fact from long input. Accuracy falls as ambiguity, distractors, or multi-step reasoning appear, and performance is uneven across the window length, even for frontier models. The job is not to stuff more tokens. The job is to decide what deserves to be in the window at each turn.

That curation spans system instructions, tool and MCP definitions, tool call results, memory, and the growing conversation history. Each has its own failure modes and optimization levers, and small choices compound over many turns.

Designing instructions, tools, and tool outputs for signal

Effective instructions avoid the extremes of vagueness and hard-coded logic. Provide concise guidance plus a few canonical examples, then iterate with measurement, for example using MLflow prompt optimization. Over-specification blocks generalization, while under-specification invites drift.

Tools are how agents offload deterministic work and reach live context. Clear names, descriptions, and schemas help the model pick correctly. Fewer, well-namespaced tools outperform many overlapping ones. Tool results are a major hallucination vector, especially in multi-turn and multi-agent setups. Return typed, validated, summarized outputs, and make errors actionable, such as hinting at a corrected column name, instead of dumping raw payloads or cryptic traces.

High-leverage patterns the speakers highlight:

• Progressive disclosure, discover many skills or Genie spaces via lightweight front matter, fully load only on activation.
• Tool search, start with a single tool that selects relevant tools at runtime rather than loading entire catalogs.
• Programmable scratchpads, let the model orchestrate tools in a Python kernel and emit only validated results, reducing back-and-forth serialization and improving accuracy and cost in code-as-action workflows.
• Conversation compaction and, when necessary, multi-agent designs that provide fresh, focused windows for complex subproblems. Use multi-agent only when single-agent limits are real, since coordination adds complexity.

Scaling with memory, evaluation, and platform guardrails

Agents without memory re-solve everything from scratch. The talk borrows human analogies: episodic memory (examples of past interactions) and semantic memory (the rules learned from them). Externalizing both improves reasoning quality and often reduces the number of steps needed to solve tasks. Keep personal and organizational memory separate with fine-grained governance in Unity Catalog.

Operationally, stateful agents benefit from low-latency storage. Lakebase, a managed Postgres with decoupled storage and compute, supports fast writes, scale-to-zero for cost control, and can serve real-time features to classical ML systems. Evaluation must be rigorous and continuous: instrument every function with MLflow tracers for fine-grained observability, use LLM-as-judge with explicit rubrics, run full regression suites against golden data for any prompt change, and watch for length bias where longer responses can fool judges.

Managed services accelerate this path. Agent Bricks handles context engineering and continuous learning for RAG, using feedback loops to improve over time. Genie and its ontology capabilities build a knowledge graph of workspace assets so text-to-SQL is less guessy and more deterministic, reinforced by business metrics, dimensions, data modeling, and trusted assets. Unity AI Gateway provides guardrails, including PII redaction, custom policies, rate and budget controls to prevent runaway agents and manage spend.

A practical path to production on the Databricks Platform

Start with built-in options like Agent Bricks, Genie, and AI functions, since much of the context work and learning is already handled. If a custom agent is needed, prototype in AI Playground, add tools and MCPs, export working code, then deploy as an app. Focus on the harness around the model, not just the model choice.

Make context an engineered product

Reliable enterprise agents emerge when context is treated as a designed, measured, and governed asset. Curate what enters the window, externalize memory, validate tool I/O, and enforce evaluation and guardrails. Model choice matters, but the surrounding harness now drives quality, cost, and scale.