SESSION

Department of Energy's Project Alexandria: A Digital Library

OVERVIEW

EXPERIENCEIn Person
TYPEBreakout
TRACKData Lakehouse Architecture
INDUSTRYEnergy and Utilities, Public Sector
TECHNOLOGIESAI/Machine Learning, Delta Lake, GenAI/LLMs
SKILL LEVELBeginner
DURATION40 min

The Department of Energy and NNSA have identified a common problem that all their experiments must solve: data management and data collection. Common practice is to leave that up to the individual projects and allow them to make their own choices. However, it was found that significant time and resources were being spent on these efforts. To alleviate this, the DOE and NNSA are working together on a full data management platform called Project Alexandria. Alexandria is meant to be a centralized repository for NA-22 project knowledge. With a federated approach to data storage, Alexandria promises to enable projects to focus on the science and let them handle the data management. Learn how we developed the infrastructure and architecture to support this massive endeavor and the common problems and gotchas all data management platforms face. Utilizing Databricks and Generative AI, we have enabled researchers not only to compute but also to curate their data effectively.

SESSION SPEAKERS

John Darrington

/Lead Digital Architect
Idaho National Laboratory