Bringing MegaBlocks to Databricks
At Databricks, we’re committed to building the most efficient and performant training tools for large-scale AI models. With the recent release of DBRX...
Page 1
Turbocharged Training: Optimizing the Databricks Mosaic AI Stack With FP8
At Databricks, we believe that the best companies in the world, in every sector, will have AI-powered systems that are trained and customized...
How We Trained Stable Diffusion for Less than $50k (Part 3)
In our previous blog post, we showed how we used the MosaicML platform, Streaming datasets, and the Composer library to train a Stable...
Training Stable Diffusion from Scratch for <$50k with MosaicML (Part 2)
We've replicated Stable Diffusion 2 for less than $50k, and we've open-sourced the training code so you can too! This is a 3x...
Farewell, CUDA OOM: Automatic Gradient Accumulation
With automatic gradient accumulation, Composer lets users seamlessly change GPU types and number of GPUs without having to worry about batch size. CUDA...