VisionHeight runs real-time queries ~3x faster, for 90% less

VisionHeight is a pre-attack intelligence platform that helps cybersecurity teams detect threats before they strike, continuously mapping adversary infrastructure so customers can query live data and get answers in seconds. That real-time experience ran on a separate Apache Pinot system — the one workload not yet on Databricks. By replacing Pinot with Lakehouse//RT, VisionHeight cut infrastructure costs over 90%, reached sub-second response times, and moved to production in days with zero data migration.

Real-time analytics for a cybersecurity platform

VisionHeight's customer-facing query interface demands interactive response times of two to four seconds across hundreds of query patterns, and no two queries are alike. "We don't know which questions the user is going to ask," said Nitsan Daniel, CTO and Co-founder of VisionHeight. "There are probably hundreds or thousands of different ways the query can access the data, and we need super fast performance because it's interactive."

To serve those queries, VisionHeight used Apache Pinot. The architecture worked, but it carried constant operational overhead: a 24/7 cluster, pipelines to copy data from Databricks, schema changes to match Pinot's format, and separate infrastructure to manage. "We had to handle copying the data, adjusting the schema, the syntax, and then deal with high availability and what happens when a machine breaks," Daniel said. "Our team was spending a lot of time troubleshooting these kinds of things."

The cost of that overhead was engineering time. "If we continued with Pinot, we had to manage it ourselves," said Erez Weiss, Software Engineer Team Lead at VisionHeight. "It was live 24/7, and every time something broke, I was spending time troubleshooting instead of building."

Zero data copying and production-ready in three days

VisionHeight evaluated several real-time platforms — popular open source columnar databases, purpose-built NewSQL engines, managed relational databases, and the real-time capabilities of other major cloud data warehouses. None met the company's performance and cost requirements without significant additional infrastructure investment.

"They weren't able to meet our KPIs for both performance and cost," Daniel said. "We would have had to spend a lot of money to get enough resources, and even then there was the overhead of copying data outside and getting to know a different platform."

The turning point came when Lakehouse//RT entered its preview program. Because VisionHeight's data was already stored in Databricks, the team could test immediately. They ran their most common queries on Lakehouse//RT (powered by the Reyden engine), then worked with the Databricks engineering team to fine-tune.

"We tested it on Reyden, gave it to the Databricks team to optimize, and when we saw the results were very good, we implemented it in our servers very fast," Weiss said. "Once we said it was okay, it was in production in two to three days."

Because Lakehouse//RT queries data directly where it lives in the lakehouse, the migration required no data copying, no ingestion pipelines, no new platform to learn, and no infrastructure to provision. The team simply updated their queries and pointed them at the new engine. 

More than 90% cost reduction and sub-second performance

The financial impact exceeded expectations. VisionHeight's Apache Pinot infrastructure required always-on EC2 clusters running 24/7. With Lakehouse//RT's serverless autoscaling, which bills only while queries are running, the team saw a cost reduction of more than 90%.

"I was actually willing to pay a little bit more to get rid of the operational overhead," Daniel said. "I didn't realize the cost reduction would be that dramatic."

Performance also improved. On VisionHeight's heaviest query, Lakehouse//RT delivered sub-second response times, nearly 3× faster than Apache Pinot, exceeding the standard required for customers' interactive use cases.

The biggest win was eliminating the complexity of running two separate systems. VisionHeight no longer copies data between platforms, manages a dedicated cluster, or troubleshoots schema mismatches between Databricks and an external database. Every workload — data processing, machine learning, analytics, batch jobs, and now real-time serving — runs on a single platform with a single copy of the data.

"It just made us able to have more resources, both cost-wise and human-wise, available for other things," Daniel said. "We can meet our business goals faster."

For a startup, that consolidation is especially powerful. As a Built on Databricks partner and the grand prize winner of the 2026 Built-On Databricks Startup Challenge, VisionHeight is proof that high-performing companies can build a complete data platform on Databricks without sacrificing requirements for cost, performance, or simplicity.