Data Engineering Central Podcast
A podcast dedicated to data engineering, covering news, topics, and general mayhem in the field. Hosted by Data Engineering in Real Life, it offers no-holds-barred discussions for data engineers.
Episodes
Agentic Data Engineering Is Here — But Can It Close the Loop?
AI can now write SQL, generate Python, review pull requests, and build entire applications with a few well-crafted prompts. Yet despite all of that progress, data engineering still feels oddly resistant to the full AI revolution. Pipelines break. Data models drift. Production data needs validation. Someone still has to close the loop.In this episode of the Data Engineering Central Podcast, I sat d
What Happens When a Software Engineer Builds a Company Alone?
When people think about starting a software company, they usually imagine raising venture capital, hiring engineers, and growing a team as quickly as possible.Michael Drogalis chose a different path.After helping build technology in the Kafka ecosystem, founding a startup that was ultimately acquired by Confluent, and leading product for stream processing, he walked away from big tech to see if on
The Creator of Pandas on AI, Apache Arrow, and the Future of Software Engineering
The creator of Pandas and co-creator of Apache Arrow, Wes McKinney, joins the Data Engineering Central Podcast for an in-depth conversation about how modern data engineering came to exist, where AI is taking software development, and why good engineering still matters more than ever.We start with Wes’ journey from building GoldenEye fan websites as a teenager to creating Pandas while working at a
From COBOL to Copilot: 30 Years of Data, BI, and AI with David Langer
What happens when someone who started programming on a Commodore 64 watches AI reshape the entire data industry?In this episode of the Data Engineering Central Podcast, I sit down with Dave Langer to explore nearly three decades of experience across software engineering, business intelligence, analytics, data science, and AI.Dave’s career spans COBOL programming on IBM mainframes, enterprise archi
Semantic Layers, Agents, and the Future of Analytics
In this episode of the Data Engineering Central Podcast, I sit down with David Jaitillake to explore the future of data engineering, analytics, and AI. David has spent nearly two decades working across data teams, from analyst roles in the early SQL Server days to leading teams, founding startups, serving as VP of AI at Cube, and now co-founding Quarry.We discuss why semantic layers have suddenly
The Future of the Lakehouse: Delta Lake, Rust, and Data Platforms at Scale
In this episode of the Data Engineering Central Podcast, I sit down with Ethan, a maintainer of delta-rs and an expert in modern lakehouse architecture working in the pharmaceutical industry.We discuss Ethan’s journey into tech and data engineering, the evolution of open table formats like Delta Lake and Apache Iceberg, and what it actually takes to build scalable enterprise data platforms in high
From Failure to AWS: What Actually Makes a Great Engineer
Victor Moreno went from failing out of a top CS program to becoming a senior engineer at AWS, and his story says a lot about what actually matters in software engineering today.In this conversation, we go deep into the reality behind the AI hype, what makes engineers valuable (it’s not writing more code), and why the future of the field looks very different from what most people think.We talk abou
How Real Data Engineers Think (Beyond Tools and Hype)
In this episode of the Data Engineering Central Podcast, I sit down with Yordan Ivanov, Head of Data Engineering at a growing fintech company, to talk through what it actually looks like to build and run real data platforms in production.Yordan’s story starts like many of mine, early programming, gaming, PHP, Linux servers—but what makes this conversation interesting is how he evolved from a gener
Data, AI, and DuckDB
In this episode of the Data Engineering Central Podcast, I sit down with Jacob Matson, Developer Advocate at MotherDuck, to unpack one of the most interesting shifts happening in data engineering right now.Jacob didn’t start in tech the way most people expect. He began in accounting, working with Excel and financial systems, before slowly realizing that the real problem he loved solving wasn’t fin
Why I Left Facebook to Work for Myself
In this episode of the Data Engineering Central Podcast, I sit down with Ben Rogojan to talk about the real story behind data engineering careers, Big Tech, and what’s changing right now.Ben shares how he went from working in kitchens… to data engineering… to Facebook… and eventually walking away from it all to build his own consulting business.And yeah, it wasn’t all glamorous.“I was making the s
Academic → CTO: What Actually Matters in Data (Matthew Housley)
Most companies don’t have a tooling problem. They have a foundation problem.In this episode, I sit down with Matthew Housley, a famed co-author of Data Engineering Fundamentals and former CTO of Ternary Data, to talk about what actually makes data teams successful and why so many organizations get it wrong despite having modern stacks, cloud platforms, and expensive dashboards.* Matthew’s path is
AI Isn’t Replacing Curious Developers
AI isn’t just changing how we write code. It’s changing what it even means to build software.In this episode of the Data Engineering Central Podcast, I sit down with Neil Roberts — a developer who’s been through every major wave of the web, from BASIC on an Atari to modern TypeScript, and now deep into LLMs and agentic workflows.This is not another surface-level “AI will change everything” convers
AI Is Changing Data Engineering Fast
In this episode of the Data Engineering Central Podcast, I sit down with Andreas Kretz to break down what is really happening in the industry right now. We go far beyond surface-level AI hype and talk about how data engineering actually works in the real world, what skills still matter, and where most engineers are wasting time.Andreas shares his full journey from industrial IoT and working at Bos
Most Data Teams Are Doing It Wrong
Most data teams think they’re building value. In reality, they’ve become ticket queues.In this episode, Chris Gambill explains his storied career in tech and data through the years, dealing with data at Fortune 500 company scale, and breaking out on his own.We cover career growth, what separates senior engineers from true strategic operators, and the biggest mistakes people make early on. We discu
From Industrial Data at BASF to Delta Lake Committer
In this episode, Robert Pack walks through his journey from engineering and simulation work to building large-scale data systems across 900+ plants at BASF.We break down what those systems actually looked like, including ingestion, modeling, and the realities of batch vs real-time in industrial environments.We also dive into:* AI Workflows for Developers* His work as a committer on Delta Lake* Whe
He Quit Apple After 13 Years
In this episode of Data Engineering Central, I sit down with Kevin, who spent 13 years working at Apple before walking away at the end of 2025.* Not to jump to another job.* Not to start a company.* But to take a step back from everything.Kevin shares his full journey—from growing up in the suburbs of Atlanta to building a career at Apple, and ultimately reaching the point where he could walk away
Spark, AI, and the Future of Data Engineering with Daniel Aronovich
In this episode of Data Engineering Central, I sit down with the founder of DataFlint, Daniel Aronovich, to talk about the realities of working with Apache Spark, distributed data systems, and the future of data engineering.We start with his early journey into tech—how he first discovered large-scale data systems and the lessons he learned from working with real-world Spark workloads.* The conver
DuckDB, AI, and the Future of Data Engineering
In this episode, I sit down with Matt Martin, Staff Engineer, data architect, ETL practitioner, and author of a new book on DuckDB coming soon, to talk about the past, present, and future of data engineering.Matt has spent decades building and architecting data platforms across technologies such as SQL Server, Oracle, DB2, Hadoop, Redshift, and BigQuery, and now focuses on modern tools such as Duc
What Decades in Software Engineering Teaches You
In this episode of Data Engineering Central, I sit down with a veteran Software Engineer John Crickett; with decades of experience in the industry to unpack what really matters in building a long and successful engineering career.We talk about how he first got into software, the early jobs and tools that shaped his thinking, and the massive technology shifts he’s witnessed across decades of engine
Data Engineering, AI, and Career Growth
In this episode of the Data Engineering Central Podcast, I sit down with Yuki (Yuki Kakegawa) to talk about his journey into tech, the tools and platforms he’s worked with, and where he thinks data engineering and AI are headed next.We cover:• How Yuki got into tech• Early career lessons and pivots• Tools and technologies he’s worked with over the years• How data engineering has evolved• The impac
Spark, Lakehouse & AI: A Deep Conversation with Bart Konieczny
In this episode of Data Engineering Central, I sit down with Bart Konieczny — data engineer, distributed systems expert, and well-known author in the Data and Spark ecosystem — for a deep technical conversation about modern data engineering.We cover:* How Bart got into tech and distributed systems* His journey through different engineering roles* Spark internals and why they still matter* The real
DevOps vs ClickOps with Maxine Meurer
In this episode of the Data Engineering Central Podcast, I sit down with Maxine Meurer, DevOps engineer, author, and educator behind I Love DevOps, for a wide-ranging conversation about careers, infrastructure, automation, and what it actually means to build systems that last.This isn’t a buzzword-heavy DevOps chat. It’s a grounded, honest discussion between two engineers about how people really g
The Evolution of Software, Streaming, and Data Engineering with Robin Moffatt
In this episode, I sit down with industry veteran Robin Moffatt — Sr. Principal Advisor in Streaming Data Technologies (Kafka, etc.) and a longtime voice in the data engineering community, to unpack the journey from old-school data architectures to today’s real-time streaming ecosystems. From early mainframe data processing and COBOL through the rise of Apache Kafka, streaming ETL, and event-drive
The Lakehouse Architecture: Multimodal Data, Delta Lake, and the Future of Data Engineering (with R. Tyler Croy)
In this episode of the Data Engineering Central Podcast, I sit down with R. Tyler Croy for a wide-ranging conversation on the present—and future—of modern data platforms.Tyler is a long-time open-source contributor to projects such as delta-rs. You can watch him on YouTube, read his blog, or work directly with him through his consultancy, Buoyant Data.Tyler has spent years deep in the open-source
Building the Full Data Stack and the Audience That Comes With It
In this episode of the Data Engineering Central Podcast, I sit down with Hoyt Emerson, founder of The Full Data Stack and Early Signal, for a wide-ranging conversation on data, analytics, and creating content in the tech world.We talk candidly about:* What actually matters in modern data and analytics* Why so much “data content” misses the mark* The difference between noise and real signal* What w
From Wiring Circuits to Data Pipelines
In this episode of the Data Engineering Central Podcast, I sit down with Andy Leonard — someone who’s been building systems long before “data engineering” was even a job title.Andy’s career didn’t start in software at all. It started with physical circuits, literally wiring systems as an electrician, before moving into programming, databases, and eventually decades of hands-on data engineering wor
From DBA to Data Everything
In this episode of the Data Engineering Central Podcast, I interview a Data OG, someone who’s been around the data space forever, and we talked about all things data, past, present, and future.I’m joined by Thomas Horton a longtime friend and one of the most well-rounded data professionals I know. Over the course of his career, Tom has worn just about every hat in data: developer, DBA, analyst, an
Scott Haines on the Future of Data Engineering
In this episode, I sit down with Scott Haines — O’Reilly author, Databricks MVP, and veteran of Yahoo, Nike, and Twilio — for a wide-ranging conversation on the real state of modern data engineering. We dig into open-source ecosystems, Lakehouse architectures, the evolution of Spark, streaming, what’s broken and what’s working in today’s data tooling, and the lessons Scott has learned scaling plat
Data Engineering Central Podcast - 09
Hello! A new episode of the Data Engineering Central Podcast is dropping today. We will be covering a few hot topics!* Cluster Fatigue* The Death of Open SourceGoing to be a great show, come along for the ride!Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bon
Data Engineering Central Podcast - Episode 8
This is a free preview of a paid episode. To hear more, visit dataengineeringcentral.substack.comHello! A new episode of the Data Engineering Central Podcast is dropping today, we will be covering a few hot topics!* Apache Iceberg Catalogs* new Boring Catalog* new full Iceberg support from Databricks/Unity Catalog* Databricks SQL Scripting* DuckDB coming to a Lake House near you* Lakebase from Dat
Apache Iceberg Rant.
Hello, my fair-weathered friends and readers! I am gone on vacation this week with my family, probably at this moment lying in the sand on a beach (Lord willing the creek don’t rise), not thinking of you all.Anywho, be that as it may, I didn’t want you to miss my pretty face, so here is a video of me ranting about Apache Iceberg, something I’ve had a lot of practice doing and enjoy quite thoroughl
Data Engineering Central Podcast - 07
This is a free preview of a paid episode. To hear more, visit dataengineeringcentral.substack.comIt’s time for another episode of the Data Engineering Central Podcast. In this episode, we cover …* Rust-based tool called UV to replace pip and poetry etc* Apache X-Table and the Future of the Lake House* How is AI going to affect you?Thanks for being a consumer of Data Engineering Central; your suppo
Data Engineering Central Podcast - 06
It’s time for another episode of the Data Engineering Central Podcast. In this episode, we cover …* AWS Lambda + DuckDB and Delta Lake (Polars, Daft, etc).* IAC - Long Live Terraform.* Databricks Data Quality with DQX.* Unity Catalog releases for DuckDB and Polars* Bespoke vs Managed Data Platforms* Delta Lake vs. Iceberg and UinFORM for a single table.Thanks for b… This is a public episode. If yo
Data Engineering Central Podcast - 05
In todays episode of Data Engineering Central Podcast we talk about a few hot topics, AWS S3 Tables, Databricks raising money, are Data Contracts Dead, and the Lake House Storage Format battle!It's a good one, buckle up! This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe
Data Engineering Central Podcast - 04
It’s time for another episode of the Data Engineering Central Podcast. In this episode we cover …* Apache Airflow vs Databricks Workflows* End-of-Year Engineering Planning for 2025* 10 Billion Row Challenge with DuckDB vs Daft vs Polars* Raw Data Ingestion.As usual, the full episode is available to paid subscribers, and a shortened version to you free loaders out there, don’t worry, I still love y
Data Engineering Central Podcast - 03
It’s time for another episode of Data Engineering Central Podcast, our third one! Topics in this episode …* Should you use DuckDB or Polars?* Small Engineering Changes (PR Reviews)* Daft vs Spark on Databricks with Unity Catalog (Delta Lake)* Primary and Foreign keys in the Lake HouseEnjoy! This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episod
Data Engineering Central Podcast - 02
Welcome to the Data Engineering Central Podcast —— a no-holds-barred discussion on the Data Landscape.Welcome to Episode 02In today’s episode, we will talk about the following topics from the Data Engineering perspective …* Using OpenAI’s o1 Model to do Data Engineering work* Lord Save us from more ETL tools* Rust for the small things* Hosted (SaaS) vs Build This is a public episode. If you'd like
Data Engineering Central Podcast
Welcome to the Data Engineering Central Podcast —— a no-holds-barred discussion on the Data Landscape.Welcome to Episode 01 In today’s episode we will talk about the following topics from the Data Engineering perspective …* Snowflake vs Databricks.* Is Apache Spark being replaced??* Notebooks in Production. Bad. This is a public episode. If you'd like to discuss this with other subscribers or get
Recommended
The Conspiracy Podcast
Cult of Conspiracy
Dispatches from Reality
Ghost Stories For The End Of The World
Hoporenkv Native American Podcast
The Conspiracy Files
The Snow Plow Show Prank Call Podcast
World of Prank Calls
TechnoSnobCast
Snoop Dogg - Flash Biográfico
The Moral Support Podcast
Jubal Phone Pranks from The Jubal Show
