Agile Data Engineering: Reliability and Continuous Delivery at Scale
Type
- Session
Format
- Hybrid
Track
- Data Engineering
Difficulty
- Intermediate
Room
- Moscone South | Upper Mezzanine | 155
Duration
- 35 min
Overview
With businesses competing to deliver value while growing rapidly and adapting to changing markets, it is more important than ever for data teams to support faster and reliable insights. We need to fail fast, learn, adapt, release and repeat. For us, Trusted and unified data infrastructure with standardized practices is at the crux of it all.
In this talk: we'll go over Atlassian's data engineering team organization, infrastructure and development practices:
• Team organization and roles
• Overview of our data engineering technical stack
• Code repositories and CICD setup
• Testing framework
• Development walkthrough
• Production data quality & integrity
• Alerting & Monitoring
• Tracking operational metrics (SLI/SLO, Cost)
See the best of Data+AI Summit
Watch on demand