Right Features, Right Time: Modernizing Real-Time Fraud Feature Serving on Databricks
Overview
| Experience | In Person |
|---|---|
| Track | Data Engineering & Streaming |
| Industry | Financial Services |
| Technologies | Unity Catalog, Lakebase |
| Skill Level | Intermediate |
Session Speakers
Austin Rainwater
/MLOps Engineer
Coinbase
Daniel Zhou
/Senior Staff Software Engineer
Coinbase
Full Summary
Inside Coinbase's real-time fraud feature platform on Databricks
Fraud decisions live in milliseconds. Coinbase describes how consolidating feature engineering and serving on the Databricks Platform enabled sub‑100 millisecond feature freshness, improved training‑serving consistency, and reduced streaming spend, all while simplifying operations for high‑volume, low‑latency ML.
FAQ
Micro‑batch processes events in fixed windows, so even fast windows introduce 300 to 400 milliseconds of freshness lag by processing the prior window. Real‑time mode runs continuous jobs that process events as they arrive, which is why Coinbase saw processing times fall into the 20 to 60 millisecond range.
If features used for training do not match what is served online, models degrade silently. Coinbase found streaming latency in older pipelines contributed to mismatches. Migrating definitions in place with a unified declarative framework cut skew by about half, and up to 71 percent on some models.
No. Streaming infrastructure costs fell roughly 60 percent. Micro‑batch required heavy overprovisioning to meet freshness targets, while real‑time mode achieved better latency with fewer resources and a smaller, standardized set of configurations.
AI agents analyzed existing features, grouped them by complexity, and generated rewrites in the new declarative format. Simple aggregations became concise SQL, medium complexity logic was reshaped into staged tables, and complex Python was preserved in testable modules. The agents also produced parity checks and validations.
Isolation prevents a less critical feature or model path from degrading a mission‑critical one. In fraud scenarios, it ensures a spike, slowdown, or error in a secondary workload cannot interfere with the serving path that blocks active attacks.