The Data Flowcast: Mastering Airflow for Data Engineering & AI
The 2025 State of Airflow report sheds light on how global users are adopting, evolving and innovating with Apache Airflow. With over 5,000 responses from 116 countries, the survey reveals critical insights into Airflows’ role in business operations, new use cases and what’s ahead for the community.
In this episode, Tamara Fingerlin, Developer Advocate at Astronomer, walks us through her process of analyzing survey data, key trends from the report and what to expect from Airflow 3.0.
Key Takeaways:
(02:14) The State of Airflow report combines anonymized telemetry and survey results.
(03:25) The survey received thousands of responses from many countries, showcasing global reach.
(04:49) The survey process involves multiple steps, from question selection to report creation.
(09:00) Many users expect to increase Airflow usage for revenue-generating or external use cases.
(11:04) Experienced users tend to utilize Airflow more for advanced use cases like MLOps.
(15:13) UI improvements offer enhanced navigation and error visibility.
(18:15) Architectural changes enable new capabilities like remote execution and language support.
(19:40) Long-requested features will be available in the new major release.
(21:00) Future aspirations include integrating data visualization capabilities into the UI.
Resources Mentioned:
https://www.linkedin.com/in/tamara-janina-fingerlin/
Astronomer | LinkedIn
https://www.linkedin.com/company/astronomer/
Astronomer | Website
https://www.astronomer.io
https://airflow.apache.org/
https://www.astronomer.io/airflow/state-of-airflow/
https://apache-airflow-slack.herokuapp.com/
Astronomer Presents: Exploring Apache Airflow® 3 Roadshows
https://www.astronomer.io/events/roadshow/
https://www.astronomer.io/events/roadshow/london/
https://www.astronomer.io/events/roadshow/new-york/
https://www.astronomer.io/events/roadshow/sydney/
https://www.astronomer.io/events/roadshow/san-francisco/
https://www.astronomer.io/events/roadshow/chicago/
Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
#AI #Automation #Airflow #MachineLearning