The AI Fundamentalists
A podcast about the fundamentals of safe and resilient modeling systems behind the AI that impacts our lives and our businesses.
Episodes
Exploring political bias and persuasion in LLMs with Dr. Jillian Fisher
In this episode of The AI Fundamentalists, hosts Andrew and Sid are joined by AI alignment and safety researcher Dr. Jillian Fisher to unpack the complex realities of political bias in Large Language Models. Dr. Fisher explains that bias isn't just a byproduct of noisy training data; it is also embedded directly into the architectural choices of the models, such as relying on a "majority
Metaphysics and modern AI: What is Reasoning and Thinking?
In this episode we conclude our series about Metaphysics and modern AI, we explore the definitions of consciousness, reasoning, and thinking to understand if AI possesses these traits. From examining legal accountability and the concept of personhood to analyzing human cognitive frameworks, we map out the differences between actual contemplative problem-solving and probabilistic pattern recogniti
Beyond Boosted Trees: Christoph Molnar on the Rise of Tabular Foundation Models
As the AI landscape evolves, the methods we use to process structured data are undergoing a silent revolution. Join us to explore how Tabular Foundation Models (TFMs) are challenging the decade-long reign of tree-based algorithms, why the traditional "train and predict" workflow is being replaced by "in-context learning," and what this shift means for the future of resilient mo
AI and the lost art of reading
As information sources have become abundant and attention spans have shortened in the age of AI, we take on the lost art of reading. Join us to explore why reading rates are falling, how that shift affects judgment and opportunity, and how interdisciplinary books help us see patterns across history, economics, and technology. To help us, Alisa Rusanoff, CEO of Eltech AI, joins us to share her pers
Metaphysics and modern AI: What is causality?
In this episode of our series about Metaphysics and modern AI, we break causality down to first principles and explain how to tell factual mechanisms from convincing correlations. From gold-standard Randomized Control Trials (RCT) to natural experiments and counterfactuals, we map the tools that build trustworthy models and safer AI.Defining causes, effects, and common causal structuresGestalt the
Why validity beats scale when building multi‑step AI systems
In this episode, Dr. Sebastian (Seb) Benthall joins us to discuss research from his and Andrew's paper entitled “Validity Is What You Need” for agentic AI that actually works in the real world. Our discussion connects systems engineering, mechanism design, and requirements to multi‑step AI that creates enterprise impact to achieve measurable outcomes.Defining agentic AI beyond LLM hypeLimits
2025 AI review: Why LLMs stalled and the outlook for 2026
Here it is! We review the year where scaling large AI models hit its ceiling, Google reclaimed momentum with efficient vertical integration, and the market shifted from hype to viability. Join us as we talk about why human-in-the-loop is failing, why generative AI agents validating other agents compounds errors, and how small expert data quietly beat the big models.• Google’s resurgence with Gemin
Big data, small data, and AI oversight with David Sandberg
In this episode, we look at the actuarial principles that make models safer: parallel modeling, small data with provenance, and real-time human supervision. To help us, long-time insurtech and startup advisor David Sandberg, FSA, MAAA, CERA, joins us to share more about his actuarial expertise in data management and AI. We also challenge the hype around AI by reframing it as a prediction machine a
Metaphysics and modern AI: What is space and time?
We explore how space and time form a single fabric, testing our daily beliefs through questions about free-fall, black holes, speed, and momentum to reveal what models get right and where they break. To help us, we’re excited to have our friend David Theriault, a science and sci-fi afficionado; and our resident astrophysicist, Rachel Losacco, to talk about practical exploration in space and time.
Metaphysics and modern AI: What is reality?
In the first episode of our series on metaphysics, Michael Herman joins us from Episode #14 on “What is consciousness?” to discuss reality. More specifically, the question of objects in reality. The team explores Plato’s forms, Aristotle’s realism, emergence, and embodiment to determine whether AI models can approximate from what humans uniquely experience.Defining objects via properties, percept
Metaphysics and modern AI: What is thinking? - Series Intro
This episode is the intro to a special project by The AI Fundamentalists’ hosts and friends. We hope you're ready for a metaphysics mini‑series to explore what thinking and reasoning really mean and how those definitions should shape AI research. Join us for thought-provoking discussions as we tackle basic questions: What is metaphysics and its relevance to AI? What constitutes reality? What
AI in practice: Guardrails and security for LLMs
In this episode, we talk about practical guardrails for LLMs with data scientist Nicholas Brathwaite. We focus on how to stop PII leaks, retrieve data, and evaluate safety with real limits. We weigh managed solutions like AWS Bedrock against open-source approaches and discuss when to skip LLMs altogether.• Why guardrails matter for PII, secrets, and access control• Where to place controls across p
AI in practice: LLMs, psychology research, and mental health
We’re excited to have Adi Ganesan, a PhD researcher at Stony Brook University, the University of Pennsylvania, and Vanderbilt, on the show. We’ll talk about how large language models LLMs) are being tested and used in psychology, citing examples from mental health research. Fun fact: Adi was Sid's research partner during his Ph.D. program.Discussion highlightsLanguage models struggle with cer
LLM scaling: Is GPT-5 near the end of exponential growth?
The release of OpenAI GPT-5 marks a significant turning point in AI development, but maybe not the one most enthusiasts had envisioned. The latest version seems to reveal the natural ceiling of current language model capabilities with incremental rather than revolutionary improvements over GPT-4. Sid and Andrew call back to some of the model-building basics that have led to this point to give thei
AI governance: Building smarter AI agents from the fundamentals, part 4
Sid Mangalik and Andrew Clark explore the unique governance challenges of agentic AI systems, highlighting the compounding error rates, security risks, and hidden costs that organizations must address when implementing multi-step AI processes. Show notes:• Agentic AI systems require governance at every step: perception, reasoning, action, and learning• Error rates compound dramatically in multi-st
Linear programming: Building smarter AI agents from the fundamentals, part 3
We continue with our series about building agentic AI systems from the ground up and for desired accuracy. In this episode, we explore linear programming and optimization methods that enable reliable decision-making within constraints. Show notes:Linear programming allows us to solve problems with multiple constraints, like finding optimal flights that meet budget requirementsThe Lagrange multipl
Utility functions: Building smarter AI agents from the fundamentals, part 2
The hosts look at utility functions as the mathematical basis for making AI systems. They use the example of a travel agent that doesn’t get tired and can be increased indefinitely to meet increasing customer demand. They also discuss the difference between this structured, economic-based approach with the problems of using large language models for multi-step tasks.This episode is part 2 of our s
Mechanism design: Building smarter AI agents from the fundamentals, Part 1
What if we've been approaching AI agents all wrong? While the tech world obsesses over larger language models (LLMs) and prompt engineering, there'a a foundational approach that could revolutionize how we build trustworthy AI systems: mechanism design.This episode kicks off an exciting series where we're building AI agents "the hard way"—using principles from game theory a
Principles, agents, and the chain of accountability in AI systems
Dr. Michael Zargham provides a systems engineering perspective on AI agents, emphasizing accountability structures and the relationship between principals who deploy agents and the agents themselves. In this episode, he brings clarity to the often misunderstood concept of agents in AI by grounding them in established engineering principles rather than treating them as mysterious or elusive entitie
Supervised machine learning for science with Christoph Molnar and Timo Freiesleben, Part 2
Part 2 of this series could have easily been renamed "AI for science: The expert’s guide to practical machine learning.” We continue our discussion with Christoph Molnar and Timo Freiesleben to look at how scientists can apply supervised machine learning techniques from the previous episode into their research.Introduction to supervised ML for science (0:00) Welcome back to Christoph Molnar a
Supervised machine learning for science with Christoph Molnar and Timo Freiesleben, Part 1
Machine learning is transforming scientific research across disciplines, but many scientists remain skeptical about using approaches that focus on prediction over causal understanding. That’s why we are excited to have Christoph Molnar return to the podcast with Timo Freiesleben. They are co-authors of "Supervised Machine Learning for Science: How to Stop Worrying and Love your Black Box.&quo
The future of AI: Exploring modeling paradigms
Unlock the secrets to AI's modeling paradigms. We emphasize the importance of modeling practices, how they interact, and how they should be considered in relation to each other before you act. Using the right tool for the right job is key. We hope you enjoy these examples of where the greatest AI and machine learning techniques exist in your routine today.More AI agent disruptors (0:56)Proxy
Agentic AI: Here we go again
Agentic AI is the latest foray into big-bet promises for businesses and society at large. While promising autonomy and efficiency, AI agents raise fundamental questions about their accuracy, governance, and the potential pitfalls of over-reliance on automation. Does this story sound vaguely familiar? Hold that thought. This discussion about the over-under of certain promises is for you.Show NotesT
Contextual integrity and differential privacy: Theory vs. application with Sebastian Benthall
What if privacy could be as dynamic and socially aware as the communities it aims to protect? Sebastian Benthall, a senior research fellow from NYU’s Information Law Institute, shows us how privacy is complex. He uses Helen Nissenbaum’s work with contextual integrity and concepts in differential privacy to explain the complexity of privacy. Our talk explains how privacy is not just about protectin
Model documentation: Beyond model cards and system cards in AI governance
What if the secret to successful AI governance lies in understanding the evolution of model documentation? In this episode, our hosts challenge the common belief that model cards marked the start of documentation in AI. We explore model documentation practices, from their crucial beginnings in fields like finance to their adaptation in Silicon Valley. Our discussion also highlights the important r
New paths in AI: Rethinking LLMs and model risk strategies
Are businesses ready for large language models as a path to AI? In this episode, the hosts reflect on the past year of what has changed and what hasn’t changed in the world of LLMs. Join us as we debunk the latest myths and emphasize the importance of robust risk management in AI integration. The good news is that many decisions about adoption have forced businesses to discuss their future and imp
Complex systems: What data science can learn from astrophysics with Rachel Losacco
Our special guest, astrophysicist Rachel Losacco, explains the intricacies of galaxies, modeling, and the computational methods that unveil their mysteries. She shares stories about how advanced computational resources enable scientists to decode galaxy interactions over millions of years with true-to-life accuracy. Sid and Andrew discuss transferable practices for building resilient modeling syst
Preparing AI for the unexpected: Lessons from recent IT incidents
Can your AI models survive a big disaster? While a recent major IT incident with CrowdStrike wasn't AI related, the magnitude and reaction reminded us that no system no matter how proven is immune to failure. AI modeling systems are no different. Neglecting the best practices of building models can lead to unrecoverable failures. Discover how the three-tiered framework of robustness, resilien
Exploring the NIST AI Risk Management Framework (RMF) with Patrick Hall
Join us as we chat with Patrick Hall, Principal Scientist at Hallresearch.ai and Assistant Professor at George Washington University. He shares his insights on the current state of AI, its limitations, and the potential risks associated with it. The conversation also touched on the importance of responsible AI, the role of the National Institute of Standards and Technology (NIST) AI Risk Managemen
Data lineage and AI: Ensuring quality and compliance with Matt Barlin
Ready to uncover the secrets of modern systems engineering and the future of AI? Join us for an enlightening conversation with Matt Barlin, the Chief Science Officer of Valence. Matt's extensive background in systems engineering and data lineage sets the stage for a fascinating discussion. He sheds light on the historical evolution of the field, the critical role of documentation, and the ear
Differential privacy: Balancing data privacy and utility in AI
Explore the basics of differential privacy and its critical role in protecting individual anonymity. The hosts explain the latest guidelines and best practices in applying differential privacy to data for models such as AI. Learn how this method also makes sure that personal data remains confidential, even when datasets are analyzed or hacked.Show NotesIntro and AI news (00:00) Google AI search te
Responsible AI: Does it help or hurt innovation? With Anthony Habayeb
Artificial Intelligence (AI) stands at a unique intersection of technology, ethics, and regulation. The complexities of responsible AI are brought into sharp focus in this episode featuring Anthony Habayeb, CEO and co-founder of Monitaur, As responsible AI is scrutinized for its role in profitability and innovation, Anthony and our hosts discuss the imperatives of safe and unbiased modeling syste
Baseline modeling and its critical role in AI and business performance
Baseline modeling is a necessary part of model validation. In our expert opinion, it should be required before model deployment. There are many baseline modeling types and in this episode, we're discussing their use cases, strengths, and weaknesses. We're sure you'll appreciate a fresh take on how to improve your modeling practices.Show notesIntroductions and news: why reporting and
Information theory and the complexities of AI model monitoring
In this episode, we explore information theory and the not-so-obvious shortcomings of its popular metrics for model monitoring; and where non-parametric statistical methods can serve as the better option. Introduction and latest news 0:03Gary Marcus has written an article questioning the hype around generative AI, suggesting it may not be as transformative as previously thought.This in contrast to
The importance of anomaly detection in AI
In this episode, the hosts focus on the basics of anomaly detection in machine learning and AI systems, including its importance, and how it is implemented. They also touch on the topic of large language models, the (in)accuracy of data scraping, and the importance of high-quality data when employing various detection methods. You'll even gain some techniques you can use right away to improve
What is consciousness, and does AI have it?
We're taking a slight detour from modeling best practices to explore questions about AI and consciousness. With special guest Michael Herman, co-founder of Monitaur and TestDriven.io, the team discusses different philosophical perspectives on consciousness and how these apply to AI. They also discuss the potential dangers of AI in its current state and why starting fresh instead of iterating
Upskilling for AI: Roles, organizations, and new mindsets
Data scientists, researchers, engineers, marketers, and risk leaders find themselves at a crossroads to expand their skills or risk obsolescence. The hosts discuss how a growth mindset and "the fundamentals" of AI can help.Our episode shines a light on this vital shift, equipping listeners with strategies to elevate their skills and integrate multidisciplinary knowledge. We share stories
Non-parametric statistics
Get ready for 2024 and a brand new episode! We discuss non-parametric statistics in data analysis and AI modeling. Learn more about applications in user research methods, as well as the importance of key assumptions in statistics and data modeling that must not be overlooked, After you listen to the episode, be sure to check out the supplement material in Exploring non-parametric statistics.Welcom
AI regulation, data privacy, and ethics - 2023 summarized
It's the end of 2023 and our first season. The hosts reflect on what's happened with the fundamentals of AI regulation, data privacy, and ethics. Spoiler alert: a lot! And we're excited to share our outlook for AI in 2024.AI regulation and its impact in 2024.Hosts reflect on AI regulation discussions from their first 10 episodes, discussing what went well and what didn't.Its po
Managing bias in the actuarial sciences with Joshua Pyle, FCAS
Joshua Pyle joins us in a discussion about managing bias in the actuarial sciences. Together with Andrew's and Sid's perspectives from both the economic and data science fields, they deliver an interdisciplinary conversation about bias that you'll only find here.OpenAI news plus new developments in language models. 0:03The hosts get to discuss the aftermath of OpenAI and Sam Altman
Model Validation: Performance
Episode 9. Continuing our series run about model validation. In this episode, the hosts focus on aspects of performance, why we need to do statistics correctly, and not use metrics without understanding how they work, to ensure that models are evaluated in a meaningful way.AI regulations, red team testing, and physics-based modeling. 0:03The hosts discuss the Biden administration's executive
Model validation: Robustness and resilience
Episode 8. This is the first in a series of episodes dedicated to model validation. Today, we focus on model robustness and resilience. From complex financial systems to why your gym might be overcrowded at New Year's, you've been directly affected by these aspects of model validation.AI hype and consumer trust (0:03) FTC article highlights consumer concerns about AI's impact on liv
Digital twins in AI systems
Episode 7. To use or not to use? That is the question about digital twins that the fundamentalists explore. Many solutions continue to be proposed for making AI systems safer, but can digital twins really deliver for AI what we know they can do for physical systems? Tune in and find out.Show notesDigital twins by definition. 0:03Digital twins are one-to-one digital models of real-life products, s
Fundamentals of systems engineering
Episode 6. What does systems engineering have to do with AI fundamentals? In this episode, the team discusses what data and computer science as professions can learn from systems engineering, and how the methods and mindset of the latter can boost the quality of AI-based innovations.Show notes News and episode commentary 0:03ChatGPT usage is down for the second straight month.The importance of und
Synthetic Data in AI
Episode 5. This episode about synthetic data is very real. The fundamentalists uncover the pros and cons of synthetic data; as well as reliable use cases and the best techniques for safe and effective use in AI. When even SAG-AFTRA and OpenAI make synthetic data a household word, you know this is an episode you can't miss.Show notesWhat is synthetic data? 0:03Definition is not a succinct one-
Modeling with Christoph Molnar
Episode 4. The AI Fundamentalists welcome Christoph Molnar to discuss the characteristics of a modeling mindset in a rapidly innovating world. He is the author of multiple data science books including Modeling Mindsets, Interpretable Machine Learning, and his latest book Introduction to Conformal Prediction with Python. We hope you enjoy this enlightening discussion from a model builder's poi
Why data matters | The right data for the right objective with AI
Episode 3. Get ready because we're bringing stats back! An AI model can only learn from the data it has seen. And business problems can’t be solved without the right data. The Fundamentalists break down the basics of data from collection to regulation to bias to quality in AI. Introduction to this episodeWhy data matters.How do big tech's LLM models stack up to the proposed EU AI Act?Ho
Truth-based AI: LLMs and knowledge graphs - back to basics
Truth-based AI: Large language models (LLMs) and knowledge graphs - The AI Fundamentalists, Episode 2Show NotesWhat’s NOT new and what is new in the world of LLMs. 3:10 Getting back to the basics of modeling best practices and rigor.What is AI and subsequently LLM regulation going to look like for tech organizations? 5:55Recommendations for reading on the topic.Andrew talks about regulation, monit
Why AI Fundamentals? | AI rigor in engineering | Generative AI isn't new | Data quality matters in machine learning
The AI Fundamentalists - Ep1 SummaryWelcome to the first episode. 0:03Welcome to the first episode of the AI Fundamentalists podcast.Introducing the hosts.Introducing Sid and Andrew. 1:23Introducing Andrew Clark, co-founder and CTO of Monitaur.Introduction of the podcast topic.What is the proper rigorous process for using AI in manufacturing? 3:44Large language models and AI.Rigorous systems for m
Recommended
Dubs Talk: A Golden State Warriors Podcast
A Life Engineered
پادکست بهزاد بلور | Behzad Bolour's Podcast
Beauty of Real Love
Brazo Wa Afrika Addictive Sessions
The Rabbit Hole: Conspiracy Theories
Detection Dispatch (Alex's Version)
Native Plants, Healthy Planet
The Conspiracy Podcast
Cult of Conspiracy
Dispatches from Reality
Ghost Stories For The End Of The World
