Chain of Thought

The AI landscape often pulls us between the allure of cutting-edge models and the quiet necessity of foundational work—yet how do these extremes actually connect to deliver value?

Join Conor Bronsdon as he welcomes Denny Lee, a self-proclaimed "data nerd" and Product Management Director, Developer Relations at Dataricks, to unpack this very spectrum, from AI's core infrastructure to its most advanced applications. Denny explains why robust logging, tracing, and data lineage are indispensable for credible AI evaluation and feedback, ultimately making AI systems more affordable, accessible, and impactful.

The discussion ventures into strategies for democratizing AI, exploring the "GenAI ladder" from efficient inference and retrieval-augmented generation to deciding when to fine-tune or pre-train models. Denny also tackles the industry's pressing hardware bottlenecks, the critical role of open standards, and the imperative of navigating data privacy in an increasingly AI-driven world. Listen for grounded advice on moving beyond the hype and making practical, value-driven decisions in your AI journey.

Chapters

00:00 Introduction and Guest Welcome

01:31 Diving into AI Foundations

02:25 Importance of Logging and Tracing

08:40 Challenges in Data Quality and Lineage

14:49 Strategies for Cost-Effective AI

19:52 Partnerships and Collaborative Opportunities

22:10 Hardware Bottlenecks in AI

24:56 China's Power and Networking Advantage

25:26 Nvidia's Super Chip and Network Fabrics

26:39 The Growing Demand for Power in AI

29:26 Practical Advice for Data Governance

35:47 Understanding Privacy in AI

36:25 Differential Privacy and Its Challenges

41:57 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website: Databricks.com

Podcast: Data Brew by Databricks (available on major podcast platforms)

YouTube: @Databricks

LinkedIn: Denny Lee

Read

SemiAnalysis Blog: https://semianalysis.com/

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠

⁠Agent Leaderboard

Agentic AI exploded in 2025, but how do businesses move beyond prototypes to deploy reliable, valuable agents at scale?

Join host Conor Bronsdon and Lyzr AI CEO Siva Surendira as they discuss the complexities of building and managing AI agents for enterprises. Siva shares his journey creating Lyzr, focusing on making powerful agent frameworks accessible and trustworthy for enterprise developers. They discuss the critical hurdles businesses face, including productionization challenges, ensuring responsible AI, and bridging the gap between rapid innovation and the stringent requirements of regulated industries.

Listen as Siva explains Lyzr's approach to embedding safety guardrails natively and learn about the nuances of multi-agent orchestration, including managerial, DAG, and hybrid flows. Siva also offers insights into the limitations of "vibe coding" for enterprise use cases and stresses the crucial role of robust evaluation (evals) and choosing the right models—from local open-source options to frontier LLMs. Explore the bottlenecks hindering adoption, like custom application integration and data readiness, and learn why Siva believes the biggest opportunity for agent companies may not lie in replacing SaaS platforms but rather in automating the mundane work currently performed by humans.

Chapters

00:22 Introduction and Guest Welcome

00:52 Enterprise Agent Framework

02:48 Building Enterprise-Friendly AI Frameworks

04:56 Enterprise Concerns with Vibe Coding

09:23 Safe and Responsible AI Implementation

11:05 Multi-Agent Orchestration

14:13 Challenges in Multi-Agent Systems

14:22 Enterprise Integration Bottlenecks

17:37 The Role of Low-Code and No-Code Solutions

19:55 Inter-Agent Communication Standards

21:49 Future of AI Agents in Enterprises

29:37 Evaluating AI Agents

36:34 Conclusion and Final Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website: lyzr.ai

LinkedIn: Siva Surendira

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Agent Leaderboard

Are we on the verge of removing all language barriers with AI?

Olga Beregovaya, VP of AI at Smartling, joins host Conor Bronsdon to tackle this question, discussing the evolution from rule-based NLP to today's powerful LLMs. Together, they confront the persistent challenges that stand in the way, like the English-centric nature of AI, domain-specific inaccuracies, and the unpredictability of model hallucinations. Olga unpacks the difficulties faced when striving for accurate, nuanced translation across all languages, especially under-resourced ones.

Beyond these hurdles, the conversation explores the cutting-edge opportunities and technical innovations driving progress, including RAG, the rise of purpose-built models, agentic AI workflows, and the potential of multilingual multimodality. Olga shares insights into boosting translator productivity, achieving more predictable quality, and the path toward human parity in translation, examining how technology and human expertise will shape the future of global communication.

Chapters

00:00 Introduction and Guest Welcome

01:14 Evolution of NLP: From Rule-Based to Machine Learning

02:40 Challenges in AI Translation

04:21 Biases in Language Models

05:28 Inference Time and Latency

05:44 English-Centric AI Models

08:53 Opportunities in AI Translation

09:14 Industries Benefiting from Language AI

10:36 Human-in-the-Loop Translation

12:06 Architectural Innovations in Language AI

16:20 Success with RAG Architectures

17:58 Multilingual Vectorization

19:54 Agentic AI in Translation

24:35 Data Sets and Data Privacy

28:30 Using Smaller, Purpose-Built Models

32:10 Future of AI in Translation

36:37 Conclusion and Farewell

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

LinkedIn Olga Beregovaya

LinkedIn ⁠Smartling

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

What if you could turn a requirement document into a full enterprise application in just minutes?

Rodrigo Coutinho, co-founder and AI Product Manager at OutSystems, joins hosts Conor Bronsdon and Atin Sanyal to explore this new reality of AI-driven development. Rodrigo shares insights from OutSystems' nearly 25-year journey, detailing their early adoption of AI and the development of their AI platform, Mentor. Discover how the pairing of AI and low-code empowers developers, accelerates the creation of enterprise applications, and shortens the cycle from idea to deployment.

But this newfound speed brings its own set of challenges. The discussion addresses the hurdles of managing AI-generated code, contrasting experiences with traditional versus low-code approaches. Learn why a dev's focus pivots from syntax to strategy, pinpointing human creativity and ideation as the crucial limiter in today's development lifecycle.

Chapters

00:00 Welcoming Rodrigo Coutinho of OutSystems

01:30 OutSystems' Early AI Journey (Pre-LLM)

03:30 The LLM Revolution & OutSystems Mentor Emerges

07:30 The Critical Need for Validating AI-Generated Apps

12:00 The Shifting Role of the Modern Developer

13:30 Quality Control & Accountability in the AI Era

16:00 Low-Code's Edge in AI Validation

18:30 OutSystems Mentor: A Deeper Look

23:30 Choosing the Right AI Models (In-House vs Public)

27:30 Future Opportunities: Speed, Experimentation & Multimodal AI

37:00 The Use Case Hurdle & Final Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website www.outsystems.com

OutSystems Mentor

LinkedIn Rodrigo Sousa Coutinho

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Generative AI dominates the conversation, but does it actually make it easier to build, lead, and sustain high-performing engineering teams?

Host Conor Bronsdon sits down with Charity Majors, co-founder and CTO of Honeycomb (.io), and the mind behind charity.wtf. Known for her sharp insights and unfiltered opinions, Charity kicks off the discussion by expanding on her popular article: 'Generative AI is not going to build your engineering team for you.' Together, they explore how AI has altered the dynamics for engineering teams and leaders. 

The discussion navigates the complex dynamics of hiring in an AI-enabled era, challenging the "senior-only" trend and championing the vital role of junior engineers in creating learning organizations. Charity also explains why writing code is often the "easy part" compared to the full lifecycle of owning and operating systems, a challenge amplified by AI-generated code. 

Finally, Conor and Charity discuss the risk of "cognitive decay" from over-reliance on AI tools and why fostering deep system understanding remains paramount for engineers and leaders.

Chapters

00:00 Introduction and Guest Welcome

01:51 Generative AI and Engineering Teams

02:26 The Writing Process and Inspiration

03:49 AI's Impact on Hiring and Team Building

05:30 Embracing AI and Automation

07:43 The Role of Junior Engineers

09:33 Building Effective Engineering Teams

17:01 Future of AI in Code Generation

20:07 High Performing Engineering Teams

21:48 Evolving Expectations for Engineering Managers

22:41 Cognitive Decay

25:00 Feedback Loops in Software Systems

26:56 Hiring for Potential vs. Experience

29:17 The Future of Observability

39:50 Closing Thoughts and Advice for Engineers

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Follow Charity: charity.wtf

Learn more about Honeycomb: www.honeycomb.io

Read: Generative AI is not going to build your engineering team for you

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Building trustworthy, scalable AI isn't just about models; it's about navigating a complex ecosystem of tools and regulations. 

Join hosts Conor Bronsdon and Atindriyo Sanyal as they explore these challenges with Dr. Maryam Ashoori, Head of Product for watsonx AI at IBM. To meet these challenges, Maryam explains how watsonx simplifies the AI stack, automates pipelines, and empowers enterprises to scale their AI operations while optimizing costs rapidly.

Maryam also explores IBM's strategy for leveraging open-source and commercial models, enabling the potential of agentic systems. Plus, she shares insights from a recent survey of 1,000 developers, revealing key takeaways about the current landscape for enterprise AI implementation, and what results mean for both developers and the enterprises they support.

Chapters

00:00 Introducing Dr. Maryam Ashoori

01:13 Overview of IBM's AI Strategy

01:47 Enterprise AI Challenges and Solutions

04:40 IBM's Approach to AI Models and Tooling

09:52 Simplifying the AI Stack

12:20 Challenges in Agentic AI

15:55 Importance of Data Management and Lineage

21:11 IBM's Strategy for Gen AI Products

23:43 Scaling Challenges with Agents

27:40 Effective Agent Evaluation Systems

35:18 Gaps and Opportunities in AI Tooling

41:35 Success Stories with watsonx

44:00 Closing Remarks

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

watsonx.ai

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

What if everyone in your organization had equal information at all times? Would meetings even exist? 

This week, we dive into the concept of information symmetry with Manoj Agarwal, co-founder and president of DevRev. Manoj, along with hosts Conor Bronsdon and Yash Sheth, explores how DevRev is connecting data, personalizing schemas, and automating complex tasks, offering a glimpse into the next generation of AI-driven workflows. This is revolutionizing enterprise data and decision-making by breaking down the silos that create information asymmetry.

Learn how AI is reshaping business outcomes and collaboration, moving us closer to a world where everyone has the information they need.

Chapters:

00:00 Welcome to Chain of Thought

00:57 Information Symmetry in Enterprises

02:03 Challenges of Decision Making

03:41 Recency Bias and Product Management

04:58 Data Silos and Information Waste

05:23 Structured vs. Unstructured Data

06:04 Collaboration and Data Retrieval Issues

08:17 DevRev's Approach to AI and Data Integration

09:23 Building a Business-Centric Knowledge Graph

10:00 Conversational AI and Automation

12:57 Agentic Interactions and Skills Programming

20:05 Multi-Agent Systems and Future Vision

21:25 Challenges in Multi-Agent Communication

25:10 Data Cleanliness and Governance

28:14 Trust and Reliability in AI Systems

36:58 Conclusion and Future Outlook

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

devrev.ai

DevRev University

LinkedIn Manoj Agarwal

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Is the agentic AI bubble about to burst? Kelly Vaughn, Director of Engineering at Spot AI, questions whether the agent craze is overpromising potential and leading startups down a path of unsustainable expectations.

Never one to shy away from a hot take, Kelly joins host Conor Bronsdon for a pragmatic look at AI, discussing the differences between building AI-enabled and traditional software, why replacing humans with AI teams will backfire (looking at you customer service), and the proliferation of AI tools.

Kelly also shares insights on constructing AI teams, navigating data governance, and building user trust while avoiding common startup pitfalls.

Chapters:

00:00 Introduction and Guest Welcome

01:13 Is Agentic AI a Bubble?

02:40 Startup Challenges and Market Noise

11:02 Building AI Products vs. Traditional Software

17:31 Ethical Implications and Governance

19:48 Constructing AI-Enabled Teams

22:07 AI Tooling and Productivity

26:00 Questioning Productivity Claims

32:28 Conclusion and Final Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠

Follow⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Kelly’s Newsletter The Modern Leader

LinkedIn Kelly Vaughn

Check out Galileo

⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Imagine cutting your legacy code modernization timeline from years to months. It’s no longer science fiction and this week’s guest is here to tell us how. 

Rachelle Palmer, Director of Product Management at MongoDB, joins hosts Conor Bronsdon and Atindriyo Sanyal, for a discussion on the groundbreaking ways AI is modernizing legacy applications. 

At MongoDB, Rachelle's forward-deployed AI engineering team is tackling the challenge of transforming complex, outdated codebases, freeing developers from technical debt. She details how LLMs are automating tasks like improving documentation, test generation, and even business logic conversion, dramatically reducing modernization timelines from years to months. What once demanded teams of dozens can now be achieved with a small, highly efficient team.

Chapters:

00:00 Introduction and Host Welcome

00:58 Challenges in Modernizing Legacy Applications

02:52 Real-World Examples of Code Modernization

04:00 The Role of LLMs in Code Modernization

08:01 Measuring Success in AI-Powered Modernization

12:28 The Future of AI in Engineering

16:17 Evaluating Modernization Success

21:12 Returning to Your Startup Roots

29:07 Forward Deployed AI Engineers

35:36 Importance of Academic Research in AI

42:10 Conclusion and Farewell

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠ Vikram⁠⁠⁠⁠

Follow⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

⁠Rachelle PalmerMongoDBApplication Modernization Factory

Check out Galileo

⁠⁠⁠⁠⁠Try Galileo⁠⁠

This week, we're sharing a special episode courtesy of 'Dev Interrupted.' Our co-host, Galileo CEO Vikram Chatterji, recently joined theDev Interrupted team for an engaging discussion on AI strategy. We were so impressed by the conversation that we wanted to share it with our audience, and they were kind enough to let us. We hope you enjoy it!

From Dev Interrupted:

"Vikram Chatterji joins Dev Interrupted’s Andrew Zigler to discuss how engineering leaders can future-proof their AI strategy and navigate an emerging dilemma: the pressure to adopt AI to stay competitive, while justifying AI spending and avoiding risky investments.

To accomplish this, Vikram emphasizes the importance of establishing clear evaluation frameworks, prioritizing AI use cases based on business needs and understanding your company's unique cultural context when deploying AI."

Chapters:

00:00 Introduction and Special Announcement

01:14 Welcome to Dev Interrupted

01:42 Challenges in AI Adoption

03:16 Balancing Business Needs and AI

06:15 Crawl, Walk, Run Approach

10:52 Building Trust and Prototyping

13:07 AI Agents as Smart Routers

13:50 Galileo's Role in AI Development

16:25 Evaluating AI Systems

25:36 Skills for Engineering Leaders

27:35 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠

Follow⁠⁠⁠ Vikram⁠⁠⁠

Follow⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠

Follow Dev Interrupted

Podcast

Substack

LinkedIn

Follow Dev Interrupted Hosts

Andrew

Ben

Check out Galileo

⁠⁠⁠⁠Try Galileo⁠⁠

Google’s strength in AI has often seemed to get lost in the midst of OpenAI announcements or DeepSeek fervor - yet Gemini 2.0 is more than good for many tasks; it’s the model to beat - and we have the research to back it up. 

This week, Logan Kilpatrick, senior product manager at Google DeepMind, joins us to discuss Gemini’s creation story, its emergence as the premiere model in the AI race, and why the launch of Gemini 2.0 is great news for developers.

During the conversation Conor and Logan explore the exciting world of multimodal AI, Gemini's strengths in agentic use cases, and its unique approach to function calling, compositional function calling, and the seamless integration of tools like search and code execution.

They also chat about Logan’s vision for a future where AI interacts with the world more naturally, offering a view of the potential of vision-first AI agents, and why Google's hardware advantage is enabling Gemini's impressive performance and long context capabilities. 

Follow along with the discussion using Galileo’s AI Agent Leaderboard:https://huggingface.co/spaces/galileo-ai/agent-leaderboard

Chapters:00:00 DeepMind's Role in Gemini's Development

03:49 Gemini 2.0 Updates and Developer Highlights

06:08 Agentic Use Cases and Function Calling

11:29 Multimodal Capabilities

16:15 Putting AI in Production

21:06 Gemini's Differentiation and Hardware

31:22 Future Vision for Gemini and G Suite Integration

35:23 Gemini for Developers

39:02 Conclusion and Farewell

Follow the hosts

Follow⁠⁠⁠Atin⁠⁠⁠

Follow⁠⁠⁠Conor⁠⁠

Follow⁠Vikram⁠

Follow⁠⁠⁠Yash⁠⁠⁠

Follow Logan

Twitter:@OfficialLoganK

LinkedIn:https://www.linkedin.com/in/logankilpatrick/

Show Notes

Try Gemini for yourself:gemini.google.com

Gemini for Developers:aistudio.google.com

Check out Galileo

⁠⁠Try Galileo⁠⁠

This week, hosts Conor Bronsdon and Atindriyo Sanyal discuss the fallout from DeepSeek's groundbreaking R1 model, its impact on the open-source AI landscape, and how its release will impact model development moving forward. They also discuss what effect (if any) export controls have had on AI innovation and whether we’re witnessing the rise of “Agents as a Service”. 

To tackle the increasing complexity of agentic systems, Conor and Atin highlight the need for robust evaluation frameworks, discussing the challenges of measuring agent performance, and how the recent launch of Galileo's agentic evaluations are empowering developers to build safer and more effective AI agents.

Chapters: 00:00 Introduction

02:09 DeepSeek's Impact and Innovations

03:43 Open Source AI and Industry Implications

13:44 Export Controls and Global AI Competition

18:55 Software as a Service

19:29 Agentic Evaluations

25:14 Metrics for Success

31:34 Conclusion and Farewell Follow the hosts Follow ⁠⁠Atin⁠⁠ Follow ⁠⁠Conor⁠

Follow Vikram Follow ⁠⁠Yash⁠⁠

Check out Galileo

Try Galileo

Show Notes

On DeepSeek and Export Controls

Introducing Agentic Evaluations

As DeepSeek so aptly demonstrated, AI doesn’t need to be closed source to be successful.

This week, Rizel Scarlett, a Staff Developer Advocate at Block, joins Conor Bronsdon to discuss the intersections between AI, open source, and developer advocacy. Rizel shares her journey into the world of AI, her passion for empowering developers, and her work on Block's new AI initiative, Goose, an on-machine developer agent designed to automate engineering tasks and enhance productivity.

Conor and Rizel also explore how AI can enable psychological safety, especially for junior developers. Building on this theme of community, they also dive into topics such as responsible AI development, ethical considerations in AI, and the impact of community involvement when building open source developer tools.

Chapters: 00:00 Rizel's Role at Block 02:41 Introducing Goose: Block's AI Agent 06:30 Psychological Safety and AI for Developers 11:24 AI Tools and Team Dynamics 17:28 Open Source AI and Community Involvement 25:29 Future of AI in Developer Communities 27:47 Responsible and Ethical Use of AI 31:34 Conclusion Follow Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Rizel Scarlett

LinkedIn: https://www.linkedin.com/in/rizel-bobb-semple/ Website: https://blackgirlbytes.dev/

Show Notes

Learn more about Goose: https://block.github.io/goose/

"In the next three to five years, every piece of software that is built on this planet will have some sort of AI baked into it." - Atin Sanyal

Chain of Thought is back for its second season, and this episode dives headfirst into the possibilities AI holds for 2025 and beyond. Join Conor Bronson as he chats with Galileo co-founders Yash Sheth (COO) and Atindriyo Sanyal (CTO) about major trends to look for this year. These include AI finding its product "tool stack" fit, generation latency decreasing, AI agents, their potential to revolutionize code generation and other industries, and the crucial role of robust evaluation tools in ensuring the responsible and effective deployment of these agents.

Yash and Atin also highlight Galileo's focus on building trust and security in AI applications through scalable evaluation intelligence. They emphasize the importance of quantifying application behavior, enforcing metrics in production, and adapting to the evolving needs of AI development.

Finally, they discuss Galileo's vision for the future and their active pursuit of partnerships in 2025 to contribute to a more reliable and trustworthy AI ecosystem.

Chapters: 00:00 AI Trends and Predictions for 2025

02:55 Advancements in LLMs and Code Generation

05:16 Challenges and Opportunities in AI Development

10:40 Evaluating AI Agents and Applications

16:07 Building Evaluation Intelligence

23:41 Research Opportunities

29:50 Advice for Leveraging AI in 2025

32:00 Closing Remarks

Show Notes:

• Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠
• Follow Yash
• Follow Atin
• Follow Conor

"This is the time. This is the time to start building... I can't say that often enough. This is the time." - Bob van Luijt

 Join Bob van Luijt, CEO and co-founder of Weaviate as he sits down with our host Conor Bronson for the Season 2 premiere of Chain of Thought. Together, they explore the ever-evolving world of AI infrastructure and the evolution of Retrieval-Augmented Generation (RAG) architecture.

Bob's journey with Weaviate offers a compelling example of how to adapt to rapid changes in the AI landscape. He discusses the importance of understanding developer needs and building AI-native solutions, emphasizing the potential of generative feedback loops and agent architectures to revolutionize data management.

Chapters: 00:00 Welcome to Season 2

1:43 The Evolution of AI Infrastructure

04:13 Navigating Rapid Changes in AI

07:39 Generative Feedback Loops and AI Native Databases

13:26 Challenges and Opportunities in AI Production

19:03 The Importance of Documentation and Developer Experience

27:13 Future Predictions and Paradigm Shifts in AI

31:17 Final Thoughts and Encouragement to Build

Follow:

Conor Bronsdon: ⁠https://www.linkedin.com/in/conorbronsdon/⁠

Bob van Luijt: ⁠https://www.linkedin.com/in/bobvanluijt/

Weaviate: https://www.linkedin.com/company/weaviate-io/

Show notes: Learn more about Weaviate: https://weaviate.io/

Can AI assistants actually enhance human connection?

As Season 1 of Chain of Thought comes to a close, Conor Bronsdon and Vinnie Giarrusso (Twilio) explore the transformative potential of AI assistants in the workplace.

Discover how these assistants function as "async junior digital employees," taking on specific tasks and contributing to the organizational structure. But will AI assistants ultimately replace human connection? Vinnie argues the opposite is true, suggesting that AI can liberate employees from mundane tasks, allowing them to focus on building meaningful relationships and providing personalized experiences.

This thought-provoking conversation takes a philosophical turn as Vinnie explores how AI could revolutionize education while potentially disrupting traditional mentorship roles. He shares his vision for a future where AI democratizes information and empowers individuals to personalize their learning journey. Finally, learn how Twilio and Galileo are partnering to shape the future of AI and what this collaboration means for both companies.

Chain of Thought will be taking a break for the holidays, but we'll see you back here on January 8th for the start of Season 2!

Chapters: 00:00 Twilio's AI Agent Platform

06:34 Ensuring Accuracy and Trustworthiness

09:49 Challenges and Failure Modes

17:39 Future of Fully Autonomous Agents

22:18 Human-AI Collaboration and Mentorship

31:24 Education and Democratization of Information

32:58 Partnership with Galileo

39:54 Conclusion and Season Wrap-Up

Follow:

Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Vinnie Giarrusso: https://www.linkedin.com/in/vinniegiarrusso/

Show notes: Twilio Alpha: ⁠https://twilioalpha.com

OWASP GenAI: https://genai.owasp.org

This week, a panel of experts (Mehmet Murat Ezbiderli, ServiceTitan; Grant Ledford, Indeed; and Vinnie Giarrusso, Twilio) join Atin Sanyal (CTO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) to explore the challenges and opportunities of deploying GenAI at enterprise scale in a conversation that's a wake-up call for any business leader looking to harness the power of AI.

Together, Atin & Conor break down key considerations like performance, cost, and model selection, emphasizing the need for robust evaluation frameworks and a shift in developer mindset.

Atin then sits down with our panel of AI engineering experts to discuss their firsthand experiences with enterprise AI, including the trade-offs of building AI systems, the evolving tools and frameworks available, and the impact these technologies are having on their organizations.

Chapters: 00:00 Enterprise Scale Deployment

05:17 Cost, Performance, and Model Selection

08:59 Building and Integrating GenAI Systems

15:26 Emerging Enterprise Use Cases

18:12 Predictions for AI in 2025

27:28 Panel Discussion: Deploying AI at Enterprise Scale

31:19 Gen AI Solutions and Challenges

33:12 Building & Deploying Traditional Infrastructure vs GenAI Infrastructure

34:36 How to Assemble Your GenAI Stack

40:39 Today's Best GenAI Use Cases

48:15 Enterprise AI Trends for 2025

50:36 Closing Remarks and Future Outlook

Follow:

Atin Sanyal: ⁠⁠⁠https://www.linkedin.com/in/atinsanyal/⁠

Mehmet Murat Ezbiderli: https://www.linkedin.com/in/mehmet-murat-ezbiderli-b894a49/ Grant Ledford: https://www.linkedin.com/in/grant-ledford-36b146a5/ Vinnie Giarrusso: https://www.linkedin.com/in/vinniegiarrusso/

Show notes: Watch all of Productionize: https://www.galileo.ai/genai-productionize-2-0

As AI agents and multimodal models become more prevalent, understanding how to evaluate GenAI is no longer optional – it's essential. 

Generative AI introduces new complexities in assessment compared to traditional software, and this week on Chain of Thought we’re joined by Chip Huyen (Storyteller, Tép Studio), Vivienne Zhang (Senior Product Manager, Generative AI Software, Nvidia) for a discussion on AI evaluation best practices. 

Before we hear from our guests, Vikram Chatterji (CEO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) give their takes on the complexities of AI evals and how to overcome them through the use of objective criteria in evaluating open-ended tasks, the role of hallucinations in AI models, and the importance of human-in-the-loop systems.

Afterwards, Chip and Vivienne sit down with Atin Sanyal (Co-Founder & CTO, Galileo) to explore common evaluation approaches, best practices for building frameworks, and implementation lessons. They also discuss the nuances of evaluating AI coding assistants and agentic systems.

Chapters: 00:00 Challenges in Evaluating Generative AI

05:45 Evaluating AI Agents

13:08 Are Hallucinations Bad?

17:12 Human in the Loop Systems

20:49 Panel discussion begins

22:57 Challenges in Evaluating Intelligent Systems

24:37 User Feedback and Iterative Improvement

26:47 Post-Deployment Evaluations and Common Mistakes

28:52 Hallucinations in AI: Definitions and Challenges

34:17 Evaluating AI Coding Assistants

38:15 Agentic Systems: Use Cases and Evaluations

43:00 Trends in AI Models and Hardware

45:42 Future of AI in Enterprises

47:16 Conclusion and Final Thoughts

Follow: Vikram Chatterji: https://www.linkedin.com/in/vikram-chatterji/

Atin Sanyal: ⁠⁠https://www.linkedin.com/in/atinsanyal/

Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/ Chip Huyen: ⁠https://www.linkedin.com/in/chiphuyen/⁠ Vivienne Zhang: ⁠⁠https://www.linkedin.com/in/viviennejiaozhang/

Show notes: Watch all of Productionize 2.0: ⁠https://www.galileo.ai/genai-productionize-2-0⁠

The “ROI of AI” has been marketed as a panacea, a near-magical solution to all business problems.

Following that promise, many companies have invested heavily in AI over the past year and are now asking themselves, “What is the return on my AI investment?”

This week on Chain of Thought, Galileo’s CEO, Vikram Chatterji joins Conor Bronsdon to discuss AI's value proposition, from the initial hype to the current search for tangible returns, offering insights into how businesses can identify the right AI use cases to maximize their investment.

Next, we’re joined by a panel of AI experts to discuss the ROI of Enterprise AI, featuring Alex Klug, Head of Product, Data Science & AI at HP; Sriram Palapudi, Sr. Dir, ML Platform Engineering at ServiceNow; and Jay Subrahmonia, Global MD for AI Research & Products at Accenture.

Together, they explore effective implementation strategies, how to measure the returns of AI adoption in the enterprise, and why AI's ROI isn't always just about the bottom line.

Chapters: 00:00 Current State of AI Investments

03:59 Challenges and Solutions in AI Implementation

08:30 Identifying and Prioritizing AI Use Cases

10:53 Ensuring Trust and Explainability in AI

15:29 Measuring ROI and Efficiency Gains

21:10 Panel Discussion Begins

21:54 Trust and Risk Management at HP

23:27 Accenture's Approach to Operationalizing AI

26:06 ServiceNow's Trade-offs and Prioritization

31:17 Measuring the success of AI for customers

36:29 Frameworks and Best Practices

40:57 Conclusion and Final Thoughts

Follow:

Vikram Chatterji: ⁠https://www.linkedin.com/in/vikram-chatterji/ Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Alex Klug: https://www.linkedin.com/in/alex-klug-67ba3655/ Sriram Palapudi: https://www.linkedin.com/in/sriram-palapudi-11294b1/ Jay Subrahmonia: https://www.linkedin.com/in/jayashree-subrahmonia-99963a/

Show notes: Watch all of Productionize 2.0: ⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

Will 2025 be the year open-source LLMs catch up with their closed-source rivals? Will an established set of best practices for evaluating AI emerge?

This week on Chain of Thought, we break out the crystal ball and give our biggest AI predictions for 2025. Listen as Sara Hooker, VP of Research at Cohere and Head of Cohere for AI predicts a trend towards smaller, more optimized AI models; Craig Wiley, Senior Director of Product, Mosaic AI at Databricks, dives into the future of multimodal AI; and Galileo’s CEO, Vikram Chatterji, shares his predictions, including the rise of open-source LLMs.

Chapters:

00:00 Introduction

02:01 Vikram's top 3 predictions

06:19 AI and nuclear energy

08:30 Giving power back to the people

13:46 Craig's predictions

20:46 The "era of toolification"

30:38 Sara's predictions

35:07 AI safety

Follow:

Vikram Chatterji: ⁠⁠https://www.linkedin.com/in/vikram-chatterji/⁠ Yash Sheth: https://www.linkedin.com/in/yash-sheth-/ Conor Bronsdon: ⁠⁠https://www.linkedin.com/in/conorbronsdon/⁠⁠

Sara Hooker: https://www.linkedin.com/in/sararosehooker/ Craig Wiley: https://www.linkedin.com/in/craigwiley/

Show notes: Watch all of Productionize 2.0: ⁠⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠