Fast, Cheap and Easy Data Ingestion with AWS Lambda and Delta Lake

In this session we will dive into examples of how to work with Delta tables from AWS Lambdas written in Python and Rust. For many ingestion, or lightweight data processing workloads AWS Lambda provides a fast, easy, and cheap execution environment. Lambda can be easily triggered from Kinesis, SQS, Kafka, S3 Event Notifications, and more, making it a powerful tool to consider when moving from the "nightly batch" to a more event-driven data platform. Using the native Python or Rust libraries for Delta Lake we will explore the transaction log, write updates, perform table maintenance, and even query Delta tables in milliseconds from AWS Lambda. After this session attendees will be ready to incorporate Lambda into their existing AWS-based data platforms.