HomepageData + AI Summit 2024 Logo
June 10–13, 2024
San Francisco + Virtual
  • Sessions
  • 2024 Call for Presentations
Apply to speak

Colossal AI: Scaling AI Models in Big Model Era

Thursday, June 29 @2:30 PM
Attending in person? Add to your schedule ↗


The proliferation of large models based on Transformer has outpaced advances in hardware, resulting in an urgent need for the ability to distribute enormous models across multiple GPUs. Despite this growing demand, best practices for choosing an optimal strategy are still lacking due to the breadth of knowledge required across HPC, DL, and distributed systems. These difficulties have stimulated both AI and HPC developers to explore the key questions: How can training and inference efficiency of large models be improved to reduce costs? How can larger AI models be accommodated even with limited resources?


What can be done to enable more community members to easily access large models and large-scale applications? In this session, we investigate efforts to solve the questions mentioned above. Firstly, diverse parallelization is an important tool to improve the efficiency of large model training and inference. Heterogeneous memory management can help enhance the model accommodation capacity of processors like GPUs.


Furthermore, user-friendly DL systems for large models significantly reduce the specialized background knowledge users need, allowing more community members to get started with larger models more efficiently. We will provide participants with a system-level open-source solution, Colossal-AI. More information can be found at https://github.com/hpcaitech/ColossalAI.


  • Breakout


  • In Person


  • DSML: ML Use Cases / Technologies


  • Enterprise Technology, Professional Services


  • Intermediate


  • 40 min
Download session slides

Session Speakers

Headshot of James Demmel

James Demmel

Dr. Richard Carl Dehmel Distinguished Professor

University of California, Berkeley

Headshot of Yang You

Yang You

Presidential Young Professor

National University of Singapore

Don't miss this year's event!

Register now