Productionising real-world ML data pipelines (Interview)

Yetunde Dada from QuantumBlack joins Jerod for a deep dive on Kedro, a workflow tool that helps structure reproducible, scaleable, deployable, robust, and versioned data pipelines. They discuss what Kedro’s all about and how it’s “changing the landscape of data pipelines in Python”, the ins/outs of open sourcing Kedro, and how they found early success by sweating the details. Finally, Jerod asks Yetunde about her passion project: a virtual reality film which debuted at the Sundance Film Festival in January.

Join the discussion

Changelog++ members support our work, get closer to the metal, and make the ads disappear. Join today!

Sponsors:

• DigitalOcean – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog.
• Retool – Retool makes it super simple to build back-office apps in hours, not days. The tool is is built by engineers, explicitly for engineers. Learn more and try it for free at retool.com/changelog
• Brain Science – For the curious! Brain Science is our new podcast exploring the inner-workings of the human brain to understand behavior change, habit formation, mental health, and being human. It’s Brain Science applied — not just how does the brain work, but how do we apply what we know about the brain to transform our lives.
• Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.

Featuring:

• Yetunde Dada – GitHub, X
• Jerod Santo – GitHub, LinkedIn, Mastodon, X

Show Notes:

• Kedro on GitHub
• Kedro’s documentation
• Cookiecutter
• CausalNex on GitHub
• Atomu at Sundance
• QuantumBlack is hiring

Something missing or broken? PRs welcome!