SESSION

Data Mesh and Compliance in a Multi-Regional Data Lake at Atlassian

Accept Cookies to Play Video

OVERVIEW

EXPERIENCEIn Person
TYPEBreakout
TRACKData Governance
INDUSTRYEnterprise Technology
TECHNOLOGIESData Sharing, Apache Spark, Delta Lake
SKILL LEVELIntermediate
DURATION40 min
DOWNLOAD SESSION SLIDES

This session will explore how we are in the journey of implementing the Data Mesh concept at Atlassian. We'll examine the contrasts between logical and physical replication, articulating their implications and trade-offs. We will also discuss a product-agnostic replication protocol used for logical replication and how we apply schema enforcement using Data Contracts. Next, we'll explore the ramifications of empowering our Cloud customers to select the AWS Region for storing their product data, leading to a deployment of a multi-region data lake. We'll also address our strategies for regulatory compliance, including GDPR and FedRAMP, as well as sophisticated enterprise functionalities such as BYOK (Bring Your Own Key). Lastly, we'll showcase how the data lake serves as the backbone for various use cases, including Atlassian Analytics, in-product data experiences, ML training, and data exports leveraging Delta Sharing, ultimately unlocking the potential of every team.

SESSION SPEAKERS

Mike Dias

/Principal Data Platform Engineer
Atlassian

Paul Ashley

/Principal Software Engineer
Atlassian