Department of Energy's Project Alexandria: A Digital Library

The Department of Energy and NNSA have identified a common problem that all their experiments must solve: data management and data collection. Common practice is to leave that up to the individual projects and allow them to make their own choices. However, it was found that significant time and resources were being spent on these efforts. To alleviate this, the DOE and NNSA are working together on a full data management platform called Project Alexandria. Alexandria is meant to be a centralized repository for NA-22 project knowledge. With a federated approach to data storage, Alexandria promises to enable projects to focus on the science and let them handle the data management. Learn how we developed the infrastructure and architecture to support this massive endeavor and the common problems and gotchas all data management platforms face. Utilizing Databricks and Generative AI, we have enabled researchers not only to compute but also to curate their data effectively.