Data Engineering Central Podcast

In todays episode of Data Engineering Central Podcast we talk about a few hot topics, AWS S3 Tables, Databricks raising money, are Data Contracts Dead, and the Lake House Storage Format battle!

It's a good one, buckle up!

This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

It’s time for another episode of the Data Engineering Central Podcast. In this episode we cover …

* Apache Airflow vs Databricks Workflows

* End-of-Year Engineering Planning for 2025

* 10 Billion Row Challenge with DuckDB vs Daft vs Polars

* Raw Data Ingestion.

As usual, the full episode is available to paid subscribers, and a shortened version to you free loaders out there, don’t worry, I still love you though.

This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

It’s time for another episode of Data Engineering Central Podcast, our third one! Topics in this episode …

* Should you use DuckDB or Polars?

* Small Engineering Changes (PR Reviews)

* Daft vs Spark on Databricks with Unity Catalog (Delta Lake)

* Primary and Foreign keys in the Lake House

Enjoy!

This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

Welcome to the Data Engineering Central Podcast —— a no-holds-barred discussion on the Data Landscape.

Welcome to Episode 02

In today’s episode, we will talk about the following topics from the Data Engineering perspective …

* Using OpenAI’s o1 Model to do Data Engineering work

* Lord Save us from more ETL tools

* Rust for the small things

* Hosted (SaaS) vs Build

This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

Welcome to the Data Engineering Central Podcast —— a no-holds-barred discussion on the Data Landscape.

Welcome to Episode 01

In today’s episode we will talk about the following topics from the Data Engineering perspective …

* Snowflake vs Databricks.

* Is Apache Spark being replaced??

* Notebooks in Production. Bad.

This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe