Unified data governance accelerates insights
Onboarding of new team members, from days to a few minutes
Access management and audit controls
The full potential of the IoT data
“Unity Catalog accelerated our ability to make data easily and securely accessible to anyone in the company for BI and ML workloads with centralized access management and audit controls using familiar and standardized SQL grants.”
GreenFlex is a European leader in environmental management services, energy efficiency and environmental impact management. To help its customers move toward more sustainable development, decarbonization and energy efficiency, GreenFlex gathers energy consumption data from its customers and leverages machine learning to identify any consumption anomalies, and helps customers devise energy strategies that are best suited to their needs. As the amount of data the company gathers and analyzes grew exponentially over the years, GreenFlex needed a simplified way to manage and govern the data it collects and make it easily and securely accessible for data exploration, business intelligence and ML use cases. With the Databricks Data Intelligence Platform, GreenFlex was able to modernize its data infrastructure and leverage terabytes of energy consumption data to devise data-driven energy optimization strategies, helping its customers reduce their energy consumption.
Simplified data access management with Unity Catalog
GreenFlex leverages the Databricks Data Intelligence Platform to ingest terabytes of IoT data, and perform analytics and machine learning on the energy consumption data collected from its customers. However, to make data available to its data team, the company maintained three unconnected workspaces for development, staging and production workloads, and was writing data in the local Hive metastore of each workspace with a separate AWS S3 bucket. At the time, it was quite complicated and painful to deal with security and access controls over the tables in each workspace. This design also led to issues with data availability across these workspaces.
Unity Catalog helped GreenFlex simplify access management by providing a single pane of glass to manage access permissions using familiar and standardized SQL. Using a single Unity Catalog metastore for all their workspaces, GreenFlex data administrators can now define access policies once at the account level and enforce those policies across the workspaces. This design simplified data management, as the company no longer needs to manage separate AWS S3 buckets for each workspace. Also with centralized audit controls, GreenFlex data administrators can track usage of the data across workspaces and be compliance and audit ready.
“Together with Unity Catalog, Delta Lake unlocked numerous pain points we had, dramatically improving the performance of queries over terabytes of data, enabling data access auditing and time travel. With the right partitions, we are now able to take full advantage of our data and refine them from Bronze to Gold tables, and from data exploration to business intelligence and data science,” said Eric Barault, Solutions Architect and DevOps Lead at GreenFlex.
Streamlined data availability fuels innovation
Using Unity Catalog, GreenFlex has now centralized data governance and access management. This has accelerated the company’s ability to onboard new employees and grant access to data from about a week to a few minutes. As a result, GreenFlex data engineers and data scientists now have more time to spend performing value-added tasks such as experimenting with data or developing new solutions.
GreenFlex developers can also create tables from new data and manage permissions more quickly, which has simplified data operations. Whereas it used to take days to fully comprehend and perform permissions tasks, GreenFlex data administrators can now achieve that in minutes while fully mastering the impact of changes, providing the company with more time to focus on helping its customers reduce energy use and improve sustainability.
