Building Enterprise Scale Data and Analytics Platforms at Amgen
- Data Lakes, Data Warehouses and Data Lakehouses
- Gesundheitswesen und Biowissenschaften
- Moscone South | Level 2 | 202
- 35 min
Over the past few years, Amgen have developed a suite of modern enterprise platforms that have served as a core foundational capability for data & analytics transformation for our business functions. We operate in mature agile teams with a dedicated product team for each of our platforms to build reusable capabilities and integrating with business programs in line with SAFe. We have massive business impact created by our platforms be it for business teams looking to self-serve onboarding data into our Data Lake or those looking to build advanced analytics applications powered by advanced NLP, knowledge graphs, and more. Our platforms are powered by modern technologies, extensively using Databricks, AWS native services, and several open source technologies.
In this talk, we will cover the following aspects of our journey along with some best practices we have learned over time:
o Engineering best practices for building large scale Data Lakes and Analytics platforms such as team organization, DevOps toolkit and maturity frameworks, and more
o Several examples of specific business problems that required us to use leading edge technologies such as Delta Lake, knowledge graphs, advanced search, and more
o Design patterns we have adopted for paradigms spanning data lakes and data warehouses, centralized vs decentralized environments, and more
o Efficiencies we have gained through automation in areas such as Test Automation, platform monitoring, and self-service APIs
Attendees building enterprise scale data engineering solutions or interested in learning about value of enterprise platforms will benefit from learning about our journey. Technology enthusiasts will enjoy learning about specific business problems and how reusable capabilities can be built to generate more value.