Airflow in Practice with Chaim Turkel

58:10
 
Share
 

Manage episode 236820620 series 1437556
By Discovered by Player FM and our community — copyright is owned by the publisher, not Player FM, and audio streamed directly from their servers.

Apache Airflow is a system for scheduling and monitoring workflows for data engineering. Airflow can be used to schedule ETL jobs, machine learning work, and script execution. Airflow also gives a developer a high level view into the graph of dependencies for their data pipelines.

Chaim Turkel is a backend data architect at Tikal. He joins the show to discuss a case study of using Airflow to rearchitect the data engineering workflow of a complex financial application. We discussed the problems that Airflow solves and the process of porting existing workflows to Airflow.

ANNOUNCEMENTS

The post Airflow in Practice with Chaim Turkel appeared first on Software Engineering Daily.

149 episodes available. A new episode about every 0 hours averaging 57 mins duration .