NEAREST BY Join: Scaling Vector Search in Databricks Runtime

How we built vector search into Databricks as a first-class SQL join, with deep kernel optimizations in Photon and a vector index in an open storage format.