Session

Right Features, Right Time: Modernizing Real-Time Fraud Feature Serving on Databricks

Overview

ExperienceIn Person
TrackData Engineering & Streaming
IndustryFinancial Services
TechnologiesUnity Catalog, Lakebase
Skill LevelIntermediate
Fraud does not wait for batch. To block funds from moving, Coinbase models need the right features at the right time—fresh, consistent, and served with predictable low latency.This session explains Coinbase’s Databricks-based feature platform for real-time fraud ML: Real-Time Mode on Structured Streaming; declarative Feature APIs + CI/CD (features as code); AI/agent-assisted migration and feature creation; and Lakebase feature serving targeting p99 <50ms with autoscaling and workload isolation.Early testing reduced typical streaming feature freshness from ~770ms to ~100–200ms (3.8–7.7×) and suggests potential for >90% streaming infra cost reduction and up to >95% p99 freshness improvement. Moving self-built batch feature sets to a managed feature store is also driving an estimated ~25–35% productivity gain by retiring bespoke pipelines and shrinking the on-call surface. Attendees leave with a reference architecture + migration playbook across latency, reliability, and TCO.

Session Speakers

Speaker placeholderIMAGE COMING SOON

Austin Rainwater

/MLOps Engineer
Coinbase

Daniel Zhou

/Senior Staff Software Engineer
Coinbase

Full Summary

Inside Coinbase's real-time fraud feature platform on Databricks

Fraud decisions live in milliseconds. Coinbase describes how consolidating feature engineering and serving on the Databricks Platform enabled sub‑100 millisecond feature freshness, improved training‑serving consistency, and reduced streaming spend, all while simplifying operations for high‑volume, low‑latency ML.

FAQ


Micro‑batch processes events in fixed windows, so even fast windows introduce 300 to 400 milliseconds of freshness lag by processing the prior window. Real‑time mode runs continuous jobs that process events as they arrive, which is why Coinbase saw processing times fall into the 20 to 60 millisecond range.

If features used for training do not match what is served online, models degrade silently. Coinbase found streaming latency in older pipelines contributed to mismatches. Migrating definitions in place with a unified declarative framework cut skew by about half, and up to 71 percent on some models.

No. Streaming infrastructure costs fell roughly 60 percent. Micro‑batch required heavy overprovisioning to meet freshness targets, while real‑time mode achieved better latency with fewer resources and a smaller, standardized set of configurations.

AI agents analyzed existing features, grouped them by complexity, and generated rewrites in the new declarative format. Simple aggregations became concise SQL, medium complexity logic was reshaped into staged tables, and complex Python was preserved in testable modules. The agents also produced parity checks and validations.

Isolation prevents a less critical feature or model path from degrading a mission‑critical one. In fraud scenarios, it ensures a spike, slowdown, or error in a secondary workload cannot interfere with the serving path that blocks active attacks.