Chain of Thought

"In the next three to five years, every piece of software that is built on this planet will have some sort of AI baked into it." - Atin Sanyal

Chain of Thought is back for its second season, and this episode dives headfirst into the possibilities AI holds for 2025 and beyond. Join Conor Bronson as he chats with Galileo co-founders Yash Sheth (COO) and Atindriyo Sanyal (CTO) about major trends to look for this year. These include AI finding its product "tool stack" fit, generation latency decreasing, AI agents, their potential to revolutionize code generation and other industries, and the crucial role of robust evaluation tools in ensuring the responsible and effective deployment of these agents.

Yash and Atin also highlight Galileo's focus on building trust and security in AI applications through scalable evaluation intelligence. They emphasize the importance of quantifying application behavior, enforcing metrics in production, and adapting to the evolving needs of AI development.

Finally, they discuss Galileo's vision for the future and their active pursuit of partnerships in 2025 to contribute to a more reliable and trustworthy AI ecosystem.

Show Notes:

• ⁠⁠⁠⁠⁠⁠⁠Check out Galileo⁠⁠⁠⁠⁠⁠⁠

• Follow Yash
• Follow Atin
• Follow ⁠Conor⁠

Chapters:

00:00 AI Trends and Predictions for 2025

02:55 Advancements in LLMs and Code Generation

05:16 Challenges and Opportunities in AI Development

10:40 Evaluating AI Agents and Applications

16:07 Building Evaluation Intelligence

23:41 Research Opportunities

29:50 Advice for Leveraging AI in 2025

32:00 Closing Remarks

"This is the time. This is the time to start building... I can't say that often enough. This is the time." - Bob van Luijt

 Join Bob van Luijt, CEO and co-founder of Weaviate as he sits down with our host Conor Bronson for the Season 2 premiere of Chain of Thought. Together, they explore the ever-evolving world of AI infrastructure and the evolution of Retrieval-Augmented Generation (RAG) architecture.

Bob's journey with Weaviate offers a compelling example of how to adapt to rapid changes in the AI landscape. He discusses the importance of understanding developer needs and building AI-native solutions, emphasizing the potential of generative feedback loops and agent architectures to revolutionize data management.

Chapters:

00:00 Welcome to Season 2

1:43 The Evolution of AI Infrastructure

04:13 Navigating Rapid Changes in AI

07:39 Generative Feedback Loops and AI Native Databases

13:26 Challenges and Opportunities in AI Production

19:03 The Importance of Documentation and Developer Experience

27:13 Future Predictions and Paradigm Shifts in AI

31:17 Final Thoughts and Encouragement to Build

Can AI assistants actually enhance human connection?

As Season 1 of Chain of Thought comes to a close, Conor Bronsdon and Vinnie Giarrusso (Twilio) explore the transformative potential of AI assistants in the workplace.

Discover how these assistants function as "async junior digital employees," taking on specific tasks and contributing to the organizational structure. But will AI assistants ultimately replace human connection? Vinnie argues the opposite is true, suggesting that AI can liberate employees from mundane tasks, allowing them to focus on building meaningful relationships and providing personalized experiences.

This thought-provoking conversation takes a philosophical turn as Vinnie explores how AI could revolutionize education while potentially disrupting traditional mentorship roles. He shares his vision for a future where AI democratizes information and empowers individuals to personalize their learning journey. Finally, learn how Twilio and Galileo are partnering to shape the future of AI and what this collaboration means for both companies.

Chain of Thought will be taking a break for the holidays, but we'll see you back here on January 8th for the start of Season 2!

Show Notes:

• ⁠⁠⁠⁠⁠⁠⁠Watch Productionize 2.0⁠⁠⁠⁠⁠⁠⁠
• ⁠⁠⁠⁠⁠⁠Check out Galileo⁠⁠⁠⁠⁠⁠⁠
• Twilio Alpha: ⁠twilioalpha.com⁠
• OWASP GenAI: ⁠genai.owasp.org⁠
• Read: ⁠Dominik Kundel on Junior/Senior relationship with AI⁠

• Follow⁠ ⁠⁠⁠ ⁠⁠⁠⁠⁠Conor Bronsdon⁠

• Follow ⁠⁠Vinnie Giarrusso

Chapters:

00:00 Twilio's AI Agent Platform

06:34 Ensuring Accuracy and Trustworthiness

09:49 Challenges and Failure Modes

17:39 Future of Fully Autonomous Agents

22:18 Human-AI Collaboration and Mentorship

31:24 Education and Democratization of Information

32:58 Partnership with Galileo

39:54 Conclusion and Season Wrap-Up

This week, a panel of experts (Mehmet Murat Ezbiderli, ServiceTitan; Grant Ledford, Indeed; and Vinnie Giarrusso, Twilio) join Atin Sanyal (CTO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) to explore the challenges and opportunities of deploying GenAI at enterprise scale in a conversation that's a wake-up call for any business leader looking to harness the power of AI.

Together, Atin & Conor break down key considerations like performance, cost, and model selection, emphasizing the need for robust evaluation frameworks and a shift in developer mindset.

Atin then sits down with our panel of AI engineering experts to discuss their firsthand experiences with enterprise AI, including the trade-offs of building AI systems, the evolving tools and frameworks available, and the impact these technologies are having on their organizations.

Show Notes:

• ⁠⁠⁠⁠⁠⁠Watch Productionize 2.0⁠⁠⁠⁠⁠⁠
• ⁠⁠⁠⁠⁠Check out Galileo⁠⁠⁠⁠⁠⁠

• Follow⁠ ⁠⁠⁠ ⁠⁠⁠⁠Atin Sanyal⁠
• Follow⁠ ⁠⁠⁠ ⁠⁠⁠⁠⁠⁠⁠Mehmet Murat Ezbiderli⁠
• Follow ⁠⁠Grant Ledford⁠
• Follow ⁠Vinnie Giarrusso

Chapters:

00:00 Enterprise Scale Deployment

05:17 Cost, Performance, and Model Selection

08:59 Building and Integrating GenAI Systems

15:26 Emerging Enterprise Use Cases

18:12 Predictions for AI in 2025

27:28 Panel Discussion: Deploying AI at Enterprise Scale

31:19 Gen AI Solutions and Challenges

33:12 Building & Deploying Traditional Infrastructure vs GenAI Infrastructure

34:36 How to Assemble Your GenAI Stack

40:39 Today's Best GenAI Use Cases

48:15 Enterprise AI Trends for 2025

50:36 Closing Remarks and Future Outlook

As AI agents and multimodal models become more prevalent, understanding how to evaluate GenAI is no longer optional – it's essential. 

Generative AI introduces new complexities in assessment compared to traditional software, and this week on Chain of Thought we’re joined by Chip Huyen (Storyteller, Tép Studio), Vivienne Zhang (Senior Product Manager, Generative AI Software, Nvidia) for a discussion on AI evaluation best practices. 

Before we hear from our guests, Vikram Chatterji (CEO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) give their takes on the complexities of AI evals and how to overcome them through the use of objective criteria in evaluating open-ended tasks, the role of hallucinations in AI models, and the importance of human-in-the-loop systems.

Afterwards, Chip and Vivienne sit down with Atin Sanyal (Co-Founder & CTO, Galileo) to explore common evaluation approaches, best practices for building frameworks, and implementation lessons. They also discuss the nuances of evaluating AI coding assistants and agentic systems.

Show Notes:

• ⁠⁠⁠⁠⁠Watch Productionize 2.0⁠⁠⁠⁠⁠
• ⁠⁠⁠⁠Check out Galileo⁠⁠⁠⁠⁠

• Follow⁠ ⁠⁠⁠ ⁠⁠⁠Vikram Chatterji⁠⁠
• Follow⁠ ⁠⁠⁠ ⁠⁠⁠⁠Chip Huyen⁠
• Follow ⁠Vivienne Zhang

Chapters:

00:00 Challenges in Evaluating Generative AI

05:45 Evaluating AI Agents

13:08 Are Hallucinations Bad?

17:12 Human in the Loop Systems

20:49 Panel discussion begins

22:57 Challenges in Evaluating Intelligent Systems

24:37 User Feedback and Iterative Improvement

26:47 Post-Deployment Evaluations and Common Mistakes

28:52 Hallucinations in AI: Definitions and Challenges

34:17 Evaluating AI Coding Assistants

38:15 Agentic Systems: Use Cases and Evaluations

43:00 Trends in AI Models and Hardware

45:42 Future of AI in Enterprises

47:16 Conclusion and Final Thoughts

The “ROI of AI” has been marketed as a panacea, a near-magical solution to all business problems.

Following that promise, many companies have invested heavily in AI over the past year and are now asking themselves, “What is the return on my AI investment?”

This week on Chain of Thought, Galileo’s CEO, Vikram Chatterji joins Conor Bronsdon to discuss AI's value proposition, from the initial hype to the current search for tangible returns, offering insights into how businesses can identify the right AI use cases to maximize their investment.

Next, we’re joined by a panel of AI experts to discuss the ROI of Enterprise AI, featuring Alex Klug, Head of Product, Data Science & AI at HP; Sriram Palapudi, Sr. Dir, ML Platform Engineering at ServiceNow; and Jay Subrahmonia, Global MD for AI Research & Products at Accenture.

Together, they explore effective implementation strategies, how to measure the returns of AI adoption in the enterprise, and why AI's ROI isn't always just about the bottom line.

Show Notes:

• ⁠⁠⁠⁠Watch Productionize 2.0⁠⁠⁠⁠
• ⁠⁠⁠Check out Galileo⁠⁠⁠⁠

• Follow⁠ ⁠⁠⁠ ⁠⁠Vikram Chatterji⁠
• Follow⁠ ⁠⁠⁠ ⁠Alex Klug⁠
• Follow ⁠Sriram Palapudi⁠
• Follow ⁠Jay Subrahmonia

Chapters:

00:00 Current State of AI Investments

03:59 Challenges and Solutions in AI Implementation

08:30 Identifying and Prioritizing AI Use Cases

10:53 Ensuring Trust and Explainability in AI

15:29 Measuring ROI and Efficiency Gains

21:10 Panel Discussion Begins

21:54 Trust and Risk Management at HP

23:27 Accenture's Approach to Operationalizing AI

26:06 ServiceNow's Trade-offs and Prioritization

31:17 Measuring the success of AI for customers

36:29 Frameworks and Best Practices

40:57 Conclusion and Final Thoughts

Will 2025 be the year open-source LLMs catch up with their closed-source rivals? Will an established set of best practices for evaluating AI emerge?

This week on Chain of Thought, we break out the crystal ball and give our biggest AI predictions for 2025. Listen as Sara Hooker, VP of Research at Cohere and Head of Cohere for AI predicts a trend towards smaller, more optimized AI models; Craig Wiley, Senior Director of Product, Mosaic AI at Databricks, dives into the future of multimodal AI; and Galileo’s CEO, Vikram Chatterji, shares his predictions, including the rise of open-source LLMs.

Show Notes:

• ⁠⁠⁠Watch Productionize 2.0⁠⁠⁠
• ⁠⁠Check out Galileo⁠⁠⁠

• Follow⁠ ⁠Sara Hooker⁠
• Follow⁠ ⁠⁠⁠⁠Craig Wiley⁠
• Follow⁠ ⁠⁠⁠ ⁠Vikram Chatterji

Chapters:

00:00 Introduction

02:01 Vikram's top 3 predictions

06:19 AI and nuclear energy

08:30 Giving power back to the people

13:46 Craig's predictions

20:46 The "era of toolification"

30:38 Sara's predictions

35:07 AI safety

AI agents have quickly emerged as the next ‘hot thing’ in AI, but what constitutes an AI agent and do they live up to the hype?

Join Brian Raymond, founder & CEO at Unstructured.io, Bob van Luijt, co-founder & CEO at Weaviate, and João Moura, founder at crewAI as they discuss the shift to agentic workflows, dissect their architecture, and tackle real-world challenges in agent deployment. 

From data management tips to generative feedback loops, this episode is your essential guide to operationalizing agents effectively.

Show Notes:

• ⁠⁠Watch Productionize 2.0⁠⁠
• ⁠Check out Galileo⁠⁠
• Follow⁠ ⁠⁠Yash Sheth co-founder & COO - Galileo⁠
• Follow⁠ ⁠⁠Brian Raymond founder & CEO - Unstructured.io⁠
• Follow⁠ ⁠⁠⁠Bob van Luijt co-founder & CEO - Weaviate⁠
• Follow⁠ ⁠⁠⁠João Moura founder - crewAI

Chapters:

00:00 Defining AI Agents

01:16 Components of Agentic Architecture

02:16 Challenges and Solutions in Agent Deployment

03:58 Data Management and Quality Issues

05:23 Operationalizing Agents in Production

06:56 API and Security Considerations

09:04 Multimodal Information and Agentic Workflows

12:42 Future of Agentic Workflows

20:20 Best Practices for Agentic Strategies

25:30 Generative Feedback Loops

28:29 Agentic Evaluations

From ChatGPT's search engine to Google's AI-powered code generation, artificial intelligence is transforming how we build and deploy technology. 

In this inaugural episode of Chain of Thought, the co-founders of Galileo explore the state of AI, from open-source models to establishing trust in enterprise applications. Plus, tune in for a segment on the impact of the Presidential election on AI regulation. The episode culminates with an interview of May Habib, CEO of Writer, who shares practical insights on implementing generative AI at scale.

Show Notes:

• ⁠Watch Productionize 2.0⁠
• ⁠Check out Galileo⁠
• Follow ⁠Vikram⁠
• Follow ⁠Yash⁠
• Follow ⁠Atin⁠
• Follow ⁠Conor⁠
• Follow ⁠May

Chapters:

00:00 Introduction to Chain of Thought Podcast

01:27 Big News in AI: ChatGPT and Anthropic

06:34 Open Source vs Proprietary AI

12:17 The Importance of Trust in AI

20:12 Challenges in AI Development and Deployment

22:07 The Role of Human Input in AI Development

28:45 The Future of AI Regulation

34:41 Interview with May Habib co-founder & CEO at Writer

40:01 What’s Writer’s secret sauce?

43:31 Challenges in productionizing GenAI

48:08 Conclusion

We are living in the age of AI. It's transforming everything around us, from the way we work and communicate, to how we solve global challenges.

But for many, AI still feels like a black box.

Introducing Chain of Thought, the podcast for software engineers and leaders that demystifies artificial intelligence.

We’ll be joined by AI innovators, tech founders, and expert researchers such as Cohere’s Sara Hooker.

Join us as we unravel actionable strategies and practical techniques for building effective GenerativeAI applications.

Discover how AI is being productionized at companies like HP, Twilio and Databricks, as each week we’ll discuss the rapid-evolving AI industry, exploring its potential to create a more productive world, and build a better, trustworthy future.

Subscribe now to Chain of Thought wherever you get your podcasts.

Chain of Thought.  Trace the logic of innovation.