The São Paulo State Department of Education (SEDUC), which supports millions of students across Brazil’s largest public education network, modernized its data architecture with Databricks Lakebase. By replacing Apache Cassandra, SEDUC eliminated costly infrastructure, simplified development workflows and improved performance for high-volume applications. With a serverless, governed foundation, the team now delivers data to critical systems faster, more securely and at lower cost.
Managing massive scale with complex, costly infrastructure
SEDUC operates one of the largest public education systems in the world, supporting thousands of schools, hundreds of thousands of teachers and millions of students across the state of São Paulo. Its data platform must handle massive, continuous workloads, including an attendance application that writes between 15 and 22 million records every day.
Initially, the team explored using Databricks Lakehouse endpoints to support the workload. While highly effective for analytics and query-driven use cases, these endpoints were not optimized for the high-throughput, low-latency requirements of operational application workloads. As demand scaled, the team evaluated alternative approaches better suited for sustained, write-heavy traffic patterns. As a result, SEDUC turned to Apache Cassandra as a high-throughput solution for write-heavy workloads.
While Cassandra solved the performance challenge, it introduced significant operational complexity. The team was required to maintain a dedicated cluster that ran continuously solely to support the Cassandra connector.
“Maintaining a permanent cluster for a single connector felt inefficient and out of place,” said Gustavo Delfino, Senior Data Architect at SEDUC. This approach also conflicted with SEDUC’s broader strategy of adopting serverless infrastructure.
Beyond operational complexity, the total cost of ownership became a key concern. The Cassandra environment operated outside the Databricks Platform, introducing additional infrastructure, management overhead and integration complexity. SEDUC needed a more integrated, cost-effective solution that aligned with its existing architecture.
Replacing Cassandra with Lakebase for a unified, serverless approach
SEDUC ultimately turned to Databricks Lakebase to simplify its architecture and streamline the delivery of processed data to applications. Already operating a medallion architecture (Bronze, Silver and Gold) within Databricks, the team needed an efficient way to serve curated Gold-layer data to downstream systems without overloading transactional databases.
Lakebase introduced a clean reverse ETL pattern:
Data is processed and refined in the Gold layer.
Data is synchronized directly into Lakebase.
Applications consume the data from Lakebase.
This eliminated the need for external connectors, dedicated clusters and complex infrastructure management.
“The data was already in our Gold layer and syncing it to Lakebase took minimal effort,” Gustavo explained. “I didn’t need to create notebooks or configure database connectivity. It just worked.”
The implementation aligned seamlessly with SEDUC’s serverless-first strategy, allowing the team to extend its existing architecture without adding operational burden.
Lakebase also improved governance and security. With Unity Catalog managing data access and lineage, SEDUC maintained full control over sensitive information. At the same time, Lakebase eliminated the need to store credentials in notebooks — an approach that posed potential security risks. “Lakebase eliminates those connections,” Gustavo noted, strengthening the platform ’s overall security posture.
Reducing costs and accelerating engineering productivity
By replacing Cassandra with Lakebase, SEDUC achieved immediate and measurable impact:
The organization eliminated the entirety of its infrastructure costs.
SEDUC removed the need to manage always-on clusters and connector dependencies.
Teams no longer spend time maintaining clusters, troubleshooting connector issues or managing runtime environments.
Applications can reliably handle high-volume workloads without overloading transactional systems.
Just as important as the cost savings was the improvement in engineering efficiency. Instead, teams can focus on building new features and improving applications that directly serve educators and students.
“That time now goes into building features,” Gustavo said.
Performance and scalability also improved. With Lakebase serving as the delivery layer for Gold-level data, this ensures that teachers marking attendance, students submitting tasks and administrators accessing dashboards all experience consistent, real-time performance.
Looking ahead, SEDUC plans to expand its use of Lakebase to replace additional JDBC-based connections across its platform. These connections create processing bottlenecks and introduce security challenges that Lakebase can eliminate. By extending this architecture, SEDUC aims to simplify operations further, improve performance and scale its data platform to meet the evolving needs of Brazil’s largest education network.
With Databricks Lakebase, SEDUC has transformed a complex, costly architecture into a streamlined, secure and scalable foundation — one that supports millions of users while enabling faster innovation across the organization.