Go offline with the Player FM app!
Prefect with Jeremiah Lowin
Manage episode 274991531 series 2626943
Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail.
In this episode we discuss:
- Solving the “negative engineering problem”
- Learning from the complaints of data engineers at Apache Airflow
- The difficulty of having a product that serves two masters
- How COVID changed the direction of Prefect
Links:
People mentioned:
- Jim O'Shaughnessy (@jposhaughnessy)
- Patrick O’Shaughnessy (@patrick_oshag)
82 episodes
Manage episode 274991531 series 2626943
Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail.
In this episode we discuss:
- Solving the “negative engineering problem”
- Learning from the complaints of data engineers at Apache Airflow
- The difficulty of having a product that serves two masters
- How COVID changed the direction of Prefect
Links:
People mentioned:
- Jim O'Shaughnessy (@jposhaughnessy)
- Patrick O’Shaughnessy (@patrick_oshag)
82 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.