The Data Engineering Show

• Talk about Daniel’s career transition from data engineering to marketing and how his background in data engineering has been a tremendous help to his marketing competence.
• Discuss the role of AI in the evolution of data movement ensuring a faster and easier process of creating data pipelines.
• Shine light on the challenges of vector databases and structured data in AI applications.
• Delve into the future of Apache Iceberg and data lakehouses, highlighting their current challenges.
• Shares insights on the golden age of data expressing the need for more data engineers, data analysts and data practitioners in the data space.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

• Delve into challenges of data quality, how it degrades over time and the one-sided data quality checks on the “last mile” of the data supply chain.
• Talk about how Gable works through a 3-layer flow of technology which is to identify data production points, trace the data flow and communicate the impact of changes before they reach production.
• Explain why the gap between data producers and consumers need to be bridged and how Gable continues to emphasize the need for effective communication and understanding data change management across teams
• Shine light on how AI can enhance data management by extracting semantics from code and effectively manage the translation output.
• Discuss Chad’s vision for 2025 which is to help companies start to care about data and how the changes made to data affect other people.

• Gable AI website
• Chad Sanderson on LinkedIn

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

• 
  From Excel to Expert: From basic Excel tasks to a full mastery of BI tools like QlikView, Wouter has blended his technical and philosophical approaches to data to become a bona fide expert.
• Data Strategy as Transformation: Good change management principles have to be adhered to if a BI project is going to bear fruit. Focus on leadership alignment, KPI clarity, and user empowerment instead of simply implementing software. 
• Challenges of Starting Small: Wouter has some tips to offer smaller companies around bootstrapping their data journey using existing tools, practical education, and even Gen AI.
• Balancing Scales: Smaller startups compared to large enterprises face a very different set of challenges.
  
  

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

• Foundations of Data Engineering: Zach Wilson emphasizes the importance of core, tech-agnostic skills in data modeling, quality assurance, and storytelling. By sharing his experiences at Airbnb and in education, he reveals that effective data engineering hinges on creating robust data models, quality controls, and persuasive narratives rather than expertise in any single tool or language.
• Bridging Academia and Practice: Matthew Housley and Joe Reis delve into the need for better data education, emphasizing hands-on experience and data fundamentals over tool-specific training, and advocate for apprenticeships and real-world collaborations in educational settings.
• Legacy Meets Modern in Data Engineering: Krishnan Viswanathan reflects on recurring themes in data engineering and the importance of adapting legacy approaches to new data needs, underscoring the challenges and benefits of vendor-built versus in-house solutions.
  
  

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

• Automated Data Pipelines: Ryanne explains how Hoptimator allows users to create and manage data pipelines using just a simple SQL SELECT query, streamlining the process of setting up Kafka topics, Flink jobs, and schemas.
• Integration with Kubernetes: The project utilizes Kubernetes to handle infrastructure tasks, treating Kubernetes as a database for managing state. This integration simplifies the orchestration of data workflows and automates routine tasks.
• Consumer-Driven Model: Ryanne discusses the shift from a producer-driven to a consumer-driven data model, emphasizing the importance of understanding and addressing consumer needs to reduce engineering complexity and optimize data systems.
• Future of Data Engineering: The conversation touches on the ongoing experimental nature of Hoptimator and its potential to transform data engineering practices, highlighting its impact on LinkedIn's data infrastructure.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

SQL’s slow. SQL’s stupid. We hear these claims every time a new shiny tool enters the market, only to realize five years later when the hype dies down that SQL is actually a good idea. 

In this super techie episode of the Data Engineering Show, Andy Pavlo, Associate Professor at Carnegie Mellon University, joins the bros to delve into database internals and optimization. 

Andy discusses leveraging ML for autonomous database optimization, using Postgres for practical applications, tuning production databases safely, and why SQL is here to stay.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Too often expensive resources and manhours are spent on dashboards no one uses, resulting in zero ROI. Philip Philip Zelitchenko, VP of Data & Analytics at ZoomInfo met the bros to talk about adopting product management principles to ensure data projects have value, and provide an unfiltered peak into ZoomInfo’s data stack and unique tech culture. 

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Matthew Weingarten, Lead Data Engineer at Disney Streaming, talks about principles essential for data quality, cost optimization, debugging, and data modeling, as adopted by the world's leading companies.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Data engineering should be less about the stack and more about best practices. While tools may change, foundational principles will remain constant. Joseph Mercado, Senior Data Engineer at LinkedIn, is on The Data Engineering Show to talk about principles that are key to success, leveraging AI for automation, and adopting software engineering methods. 

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Joe Hellerstein is the Jim Gray Professor of Computer Science at Berkeley and Joseph Gonzalez is an Associate Professor in the Electrical Engineering and Computer Science department. 

They’ve inspired generations of database enthusiasts (including Benji and Eldad) and have come on the show to talk about all things LLM and RunLLM which they co-founded.

If you consider yourself a hardcore engineer, this episode is for you.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

There are two types of data influencers on LinkedIn:

1. Those who talk directly about the products and companies they work for
2. Those that provide more general guidance, tips and opinions 

Can influencers actually be passionate about the products they’re developing and straightforwardly talk about them without sounding salesly? 

We’re kicking off 2024 with the amazing Megan Lieu on a new Data Engineering Show episode.

Megan is one of those influencers that combine the two approaches, and with almost 100K followers, her content seems to be resonating with many data folks. 

She talked to the bros about her approach to data advocacy as well as the power of notebooks, especially when they become broader and enable collaboration.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Every data team should have at least one data engineer with a software engineering background. This time on The Data Engineering Show, Xiaoxu Gao is an inspiring Python and data engineering expert with 10.6K followers on Medium. 

She’s a data engineer at Adyen with a software engineering background, and she met the bros to talk about why both software and data engineering skills are so important.

Without software engineering skills you’ll be limited to the rigid capabilities of your stack. But without data engineering skills you’ll find it hard to be cost effective and see the bigger picture.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Vin Vashista, the guy we all love to follow, has never seen a dashboard with positive ROI. This time on The Data Engineering Show, he met the bros to talk about the difference between BI dashboards and analytics that actually introduce knowledge. It’s no longer just about the data volume, it’s about quality and relevance.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

After co-writing the best-selling book ‘Fundamentals of Data Engineering’, Joe Reis and Matt Housely joined the bros for some much-needed ranting, priceless data advice, and good laughs. So why are we still talking about providing business value and dashboards, even though we don’t really have anything new to say? If there are so many great tools in the data stack, why are we still so troubled? How can we focus more on things like data governance and data quality that’ll actually push the industry forward?

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

As people in the data industry go, Bill Inmon is among the top, often seen as the godfather of the data warehouse. In this Data Engineering Show episode, Bill Inmon talks about surviving rabbit holes throughout the evolution of data, the data modeling renaissance, and why ChatGPT is not Textual ETL.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

As companies scale, data gets messy. The data team says one thing, the business team says something completely different. Meenal Iyer, VP Data at Momentive.ai, Met the Data Bros to talk about enforcing collaboration in large organizations to ensure what she considers the three most important data factors: Adoption, Trust, and Value.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

When it comes to data management, have we come a long way since the early 2000s? Or has it simply taken us 20 years to finally realize that you can’t scale properly without data modeling. With over 20 years of experience in the data space, leading engineering teams at Cisco, Oracle, Greenplum, and now as Sr. Director of Engineering at BlackRock, Krishnan Viswanathan talks about the data engineering challenges that existed two decades ago and still exist today.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

How good you are at Spark or Flink ≠ how good you are at data engineering. After years of data engineering experience at Airbnb, Netflix, and Facebook, Zach Wilson is now focused on spreading the knowledge in EcZachly and all over social media. He met Benjamin Wagner to explain why data modeling and storytelling are more important than the actual tech, why data engineering is going to see more job growth than data science, and what brought him to start creating content, reaching over 250K followers on LinkedIn.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Data engineers are not paid to do support. Liran Yogev, Director of Engineering at ZipRecruiter, and Doron Porat, Director of Infrastructure at Yotpo talk about building resilient self-service products that keep customers happy and engineers calm. 

They walked the bros through their data stacks and explained how ZipRecruiter is completely rebuilding its data layer from scratch.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Barr Moses, CEO of Monte Carlo explains the difference between data quality and data observability, and how to make sure your data is accurate in a world where so many different teams are accessing it.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing