Elixir: The Wickedly Awesome Batch and Stream Processing Language You Should Have in Your Toolbox
- Data Engineering
- 40 min
Elixir is an Erlang-VM bytecode-compatible programming language that is growing in popularity.
In this session I will show how you can apply Elixir towards solving data engineering problems in novel ways.
• How to leverage Erlang's lightweight distributed process coordination to run clusters of workers across docker containers and perform data ingestion.
• A framework that hooks Elixir functions as steps into Airflow graphs.
• How to consume and process Kafka events directly within Elixir microservices.
For each of the above I'll show real system examples and walk through the key elements step by step. No prior familiarity with Erlang or Elixir will be required.