Leveraging Documents and Data to Create a Custom LLM Chatbot

How do you customize a LLM chatbot to address a collection of documents and data? What tools and techniques can you use to build embeddings into a vector database? This week on the show, Calvin Hendryx-Parker is back to discuss developing an AI-powered, Large Language Model-driven chat interface.

Calvin is the co-founder and CTO of Six Feet Up, a Python and AI consultancy. He shares a recent project for a family-owned seed company that wanted to build a tool for customers to access years of farm research. These documents were stored as brochure-style PDFs and spanned 50 years.

We discuss several of the tools used to augment a LLM. Calvin covers working with LangChain and vectorizing data with ChromaDB. We talk about the obstacles and limitations of capturing documentation.

Calvin also shares a smaller project that you can try out yourself. It takes the information from a conference website and creates a chatbot using Django and Python prompt-toolkit.

This episode is sponsored by Mailtrap.

Topics:

• 00:00:00 – Introduction
• 00:02:21 – Background on the project
• 00:03:51 – Complexity of adding documents
• 00:09:01 – Retrieval-augmented generation and providing links
• 00:13:46 – Updating information and larger conversation context
• 00:18:08 – Sponsor: Mailtrap
• 00:18:43 – Working with context
• 00:21:02 – Temperature adjustment
• 00:22:07 – Rally Conference Chatbot Project
• 00:26:20 – Vectorization using ChromaDB
• 00:32:49 – Employing Python prompt-toolkit
• 00:35:07 – Learning libraries on the fly
• 00:37:38 – Video Course Spotlight
• 00:39:00 – Problems with tables in documents
• 00:42:30 – Everything looks like a chat box
• 00:44:26 – Finding the right fit for a client and customer
• 00:49:05 – What are questions you ask a new client now?
• 00:51:54 – Canada Air anecdote
• 00:56:20 – How do you stay up to date on these topics?
• 01:01:03 – What are you excited about in the world of Python?
• 01:03:22 – What do you want to learn next?
• 01:04:58 – How can people follow your work online?
• 01:05:31 – IndyPy
• 01:07:13 – Thanks and goodbye

Show Links:

• Transforming Agricultural Data with AI — Six Feet Up
• Build ChatGPT-like Apps with AI — Six Feet Up
• Innovate with AI: Build ChatGPT-like Apps - YouTube
• What is retrieval-augmented generation? - IBM Research Blog
• rally-llm-presentation - sixfeetup - GitHub
• Python Prompt Toolkit 3.0 — Documentation
• Chroma - the AI-native open-source embedding database
• Embeddings and Vector Databases With ChromaDB – Real Python
• LangChain
• Build an LLM RAG Chatbot With LangChain – Real Python
• Air Canada must pay after chatbot lies to grieving passenger - The Register
• I’d Buy That for a Dollar: Chevy Dealership’s AI Chatbot Goes Rogue
• Omnivore
• TLDR AI - Get smarter about AI in 5 minutes
• Tech Brew
• Simon Willison’s Weblog
• llm: Access large language models from the command-line - simonw - GitHub
• PyCon US 2024
• Syntorial: The Ultimate Synthesizer Tutorial
• Blog — Six Feet Up
• Calvin Hendryx-Parker - LinkedIn
• Eclipse Insights: How AI is Transforming Solar Astronomy - YouTube

Level up your Python skills with our expert-led courses:

• Sneaky REST APIs With Django Ninja
• How to Work With a PDF in Python
• Command Line Interfaces in Python

Support the podcast & join our community of Pythonistas