MLOps.community

Global Feature Store: Optimizing Locally and Scaling Globally at Delivery Hero // MLOps Podcast #263 with Delivery Hero's Gottam Sai Bharath, Senior Machine Learning Engineer & Cole Bailey, ML Platform Engineering Manager. // Abstract Delivery Hero innovates locally within each department to develop MLOps practices most effective in that particular context. We also discuss our efforts to reduce redundancy and inefficiency across the company. Hear about our experiences in creating multiple micro feature stores within our departments, and our goal to unify these into a Global Feature Store that is more powerful when combined. // Bio Sai Bharath Gottam With a passion for translating complex technical concepts into practical solutions, Sai excels at making intricate topics accessible and engaging. As a Senior Machine Learning Engineer at Delivery Hero, Sai works on cutting-edge machine learning platforms that guarantee seamless delivery experiences. Always eager to share insights and innovations, Sai is committed to making technology understandable and enjoyable for all. Cole Bailey Bridging data science and production-grade software engineering. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.deliveryhero.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sai on LinkedIn: https://www.linkedin.com/in/sai-bharath-gottam/ Connect with Cole on LinkedIn: www.linkedin.com/in/cole-bailey Timestamps: [00:00] Sai and Cole's preferred coffee [00:42] Takeaways [01:51] Please like, share, leave a review, and subscribe to our MLOps channels! [02:08] Life changes in Delivery Hero [05:21] Global Feature Store and Pandora [12:21] Tech integration strategies [20:08] Defining Feature and Feature Store [22:46] Feature Store vs Data Platform [26:26] Features are discoverable [32:56] Onboarding and Feature Testing [36:00] Data consistency [41:07] Future Vision Feature Store [44:17] Multi-cloud strategies [46:33] Wrap up

Adam Kamor is the Co-founder of Tonic, a company that specializes in creating mock data that preserves secure datasets.

RAG Quality Starts with Data Quality // MLOps Podcast #262 with Adam Kamor, Co-Founder & Head of Engineering of Tonic.ai. // Abstract Dive into what makes Retrieval-Augmented Generation (RAG) systems tick—and it all starts with the data. We’ll be talking with an expert in the field who knows exactly how to transform messy, unstructured enterprise data into high-quality fuel for RAG systems. Expect to learn the essentials of data prep, uncover the common challenges that can derail even the best-laid plans, and discover some insider tips on how to boost your RAG system’s performance. We’ll also touch on the critical aspects of data privacy and governance, ensuring your data stays secure while maximizing its utility. If you’re aiming to get the most out of your RAG systems or just curious about the behind-the-scenes work that makes them effective, this episode is packed with insights that can help you level up your game. // Bio Adam Kamor, PhD, is the Co-founder and Head of Engineering of Tonic.ai. Since completing his PhD in Physics at Georgia Tech, Adam has committed himself to enabling the work of others through the programs he develops. In his roles at Microsoft and Kabbage, he handled UI design and led the development of new features to anticipate customer needs. At Tableau, he played a role in developing the platform’s analytics/calculation capabilities. As a founder of Tonic.ai, he is leading the development of unstructured data solutions that are transforming the work of fellow developers, analysts, and data engineers alike. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.tonic.ai Various topics about RAG and LLM security are available on Tonic.ai's blogs: https://www.tonic.ai/blog https://www.tonic.ai/blog/how-to-prevent-data-leakage-in-your-ai-applications-with-tonic-textual-and-snowpark-container-services https://www.tonic.ai/blog/rag-evaluation-series-validating-the-rag-performance-of-the-openais-rag-assistant-vs-googles-vertex-search-and-conversation https://www.youtube.com/watch?v=5xdyt4oRONU https://www.tonic.ai/blog/what-is-retrieval-augmented-generation-the-benefits-of-implementing-rag-in-using-llms --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/adam-kamor-85720b48/ Timestamps: [00:00] Adam's preferred coffee [00:24] Takeaways [00:59] Huge shout out to Tonic.ai for supporting the community! [01:03] Please like, share, leave a review, and subscribe to our MLOps channels! [01:18] Naming a product [03:38] Tonic Textual [08:00] Managing PII and Data Safety [10:16] Chunking strategies for context [14:19] Data prep for RAG [17:20] Data quality in AI systems [20:58] Data integrity in PDFs [27:12] Ensuring chatbot data freshness [33:02] Managed PostgreSQL and Vector DB [34:49] RBAC database vs file access [37:35] Slack AI data leakage solutions [42:26] Hot swapping [46:06] LLM security concerns [47:03] Privacy management best practices [49:02] Chatbot design patterns [50:39] RAG growth and impact [52:40] Retrieval Evaluation best practices [59:20] Wrap up

Jonathan Rioux is a Managing Principal of AI Consulting for EPAM Systems, where he advises clients on how to get from idea to realized AI products with the minimum of fuss and friction. Who's MLOps for Anyway? // MLOps Podcast #261 with Jonathan Rioux, Managing Principal, AI Consulting at EPAM Systems. // Abstract The year is 2024 and we are all staring into the cliff towards the abyss of disillusionment for Generative AI. Every organization, developer, and AI-adjacent individual is now talking about "making AI real" and "turning a ROI on AI initiatives". MLOps and LLMOps are taking the stage as the solution; equip your AI teams with the best tools money can buy, grab tokens by the fistful, and look at value raking in. Sounds familiar and eerily similar to the previous ML hype cycles? From solo devs to large organizations, how can we avoid the same pitfalls as last time and get out of the endless hamster wheel? // Bio Jonathan is a Managing Principal of AI Consulting for EPAM, where he advises client on how to get from idea to realized AI products with the minimum of fuss and friction. He's obsessed with the mental models of ML and how to organize harmonious AI practices. Jonathan published "Data Analysis with Python and PySpark" (Manning, 2022). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: raiks.ca --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jonathan on LinkedIn: https://www.linkedin.com/in/jonathanrx/ Timestamps: [00:00] Jonathan's preferred coffee [00:25] Takeaways [01:44] MLOps as not being sexy [03:49] Do not conflate MLOps with ROI [06:21] ML Certification Business Idea [11:02] AI Adoption Missteps [15:40] Slack AI Privacy Risks [18:17] Decentralized AI success [22:00] Michelangelo Hub-Spoke Model [27:45] Engineering tools for everyone [33:38 - 35:20] SAS Ad [35:21] POC to ROI transition [42:08] Repurposing project learnings [46:24] Balancing Innovation and ROI [55:35] Using classification model [1:00:24] Chatbot evolution comparison [1:01:20] Balancing Automation and Trust [1:06:30] Manual to AI transition [1:09:57] Wrap up

Shiva Bhattacharjee is the Co-founder and CTO of TrueLaw, where we are building bespoke models for law firms for a wide variety of tasks. Alignment is Real // MLOps Podcast #260 with Shiva Bhattacharjee, CTO of TrueLaw Inc. // Abstract If the off-the-shelf model can understand and solve a domain-specific task well enough, either your task isn't that nuanced or you have achieved AGI. We discuss when is fine-tuning necessary over prompting and how we have created a loop of sampling - collecting feedback - fine-tuning to create models that seem to perform exceedingly well in domain-specific tasks. // Bio 20 years of experience in distributed and data-intensive systems spanning work at Apple, Arista Networks, Databricks, and Confluent. Currently CTO at TrueLaw where we provide a framework to fold in user feedback, such as lawyer critiques of a given task, and fold them into proprietary LLM models through fine-tuning mechanics, resulting in 7-10x improvements over the base model. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: www.truelaw.ai --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Shiva on LinkedIn: https://www.linkedin.com/in/shivabhattacharjee/ Timestamps: [00:00] Shiva's preferred coffee [00:58] Takeaways [01:17] DSPy Implementation [04:57] Evaluating DSPy risks [08:13] Community-driven DSPy tool [12:19] RAG implementation strategies [17:02] Cost-effective embedding fine-tuning [18:51] AI infrastructure decision-making [24:13] Prompt data flow evolution [26:32] Buy vs build decision [30:45] Tech stack insights [38:20] Wrap up

Vikram Rangnekar is an open-source software developer focused on simplifying LLM integration. He created LLMClient, a TypeScript library inspired by Stanford's DSP paper. With years of experience building complex LLM workflows, he previously worked as a senior software engineer at LinkedIn on Ad Serving. Ax a New Way to Build Complex Workflows with LLMs // MLOps Podcast #259 with Vikram Rangnekar, Software Engineer at Stealth. // Abstract Ax is a new way to build complex workflows with LLMs. It's a typescript library based on research done in the Stanford DSP paper. Concepts such as prompt signatures, prompt tuning, and composable prompts help you build RAG and agent-powered ideas that have till now been hard to build and maintain. Ax is designed for production usage. // Bio Vikram builds open-source software. Currently working on making it easy to build with LLMs. Created Ax a typescript library that abstracts over all the complexity of LLMs, it is based on the research done in the Stanford DSP paper. Worked extensively with LLMs over the last few years to build complex workflows. Previously worked as a senior software engineer with LinkedIn on Ad Serving. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links The unofficial DSPy framework. Build LLM-powered Agents and "Agentic workflows" based on the Stanford DSP paper: https://axllm.dev All the Hard Stuff with LLMs in Product Development // Phillip Carter // MLOps Podcast #170: https://youtu.be/DZgXln3v85s --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vikram on LinkedIn: https://www.linkedin.com/in/vikramr Timestamps: [00:00] Vikram preferred coffee [00:41] Takeaways [01:05] Data Engineering for AI/ML Conference Ad [01:41] Vikram's work these days [04:54] Fine-tuned Model insights [06:22] Java Script tool evolution [16:14] DSP knowledge distillation [17:34] DSP vs Manual examples [22:53] Optimizing task context [27:58] API type validation explained [30:25] LLM value and innovation [34:22] Navigating complex systems [37:30] DSP code generators explained [40:56] Exploring LLM personas [42:45] Optimizing small agents [43:32] Complex task assistance [49:53] Wrap up

MLOps Coffee Sessions #177 with Mohamed Abusaid and Mara Pometti, Building in Production Human-centred GenAI Solutions sponsored by QuantumBlack, AI by McKinsey. // Abstract Trust is paramount in the adoption of new technologies, especially in the realm of education. Mohamed and Mara shed light on the importance of AI governance programs and establishing AI governance boards to ensure safe and ethical use of technology while managing associated risks. They discuss the impact on customers, potential risks, and mitigation strategies that organizations must consider to protect their brand reputation and comply with regulations. // Bio Mara Pometti Mara is an Associate Design Director at McKinsey & Company, where she helps organisations drive AI adoption through human-centered methods. She defines herself as a data-savvy humanist. Her practice spans across AI, data journalism, and design with the overarching objective of finding the strategic intersection between AI models and human intents to implement responsible AI systems that move organisations forward. Previously, she led the AI Strategy practice at IBM, where she also developed the company’s first-ever data storytelling program. Yet, by background, she is a data journalist. She worked as a data journalist for agencies and newsrooms like Aljazeera. Mara lectured at many universities about how to humanize AI, including the London School of Economics. Her books and writing explore how to weave a humanistic approach to AI development. Mohamed Abusaid Am Mohamed, a tech enthusiast, hacker, avid traveler, and foodie all rolled into one individual. Built his first website when he was 9 and fell in love with computers and the internet ever since. Graduated with computer science from university although dabbled in electrical, electronic, and network engineering before that. When he's not reading up on the latest tech conversations and products on Hacker News, Mohamed spends his time traveling to new destinations and exploring their cuisine and culture. Mohamed works with different companies helping them tackle challenges in developing, deploying, and scaling their analytics to reach its potential. Some topics he's enthusiastic about include MLOps, DataOps, GenerativeAI, Product thinking, and building cross-functional teams to deliver user-first products. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links QuantumBlack, AI by McKinsey: https://www.mckinsey.com/capabilities/quantumblack/how-we-help-clients --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/ Connect with Mara on LinkedIn: https://www.linkedin.com/in/mara-pometti Connect with Mohamed on LinkedIn: https://www.linkedin.com/in/mabusaid/

Markus Stoll is the Co-Founder of Renumics and the developer behind the open-source interactive ML dataset exploration tool, Spotlight. He shares insights on:

AI in Engineering and Manufacturing
Interactive ML Data Visualization
ML Data Exploration

Follow Markus for hands-on articles about leveraging ML while keeping a strong focus on data.

Visualize - Bringing Structure to Unstructured Data // MLOps Podcast #258 with Markus Stoll, CTO of Renumics. A huge thank you to SAS for their generous support! // Abstract This talk is about how data visualization and embeddings can support you in understanding your machine-learning data. We explore methods to structure and visualize unstructured data like text, images, and audio for applications ranging from classification and detection to Retrieval-Augmented Generation. By using tools and techniques like UMAP to reduce data dimensions and visualization tools like Renumics Spotlight, we aim to make data analysis for ML easier. Whether you're dealing with interpretable features, metadata, or embeddings, we'll show you how to use them all together to uncover hidden patterns in multimodal data, evaluate the model performance for data subgroups, and find failure modes of your ML models. // Bio Markus Stoll began his career in the industry at Siemens Healthineers, developing software for the Heavy Ion Therapy Center in Heidelberg. He learned about software quality while developing a treatment machine weighing over 600 tons. He earned a Ph.D., focusing on combining biomechanical models with statistical models, through which he learned how challenging it is to bridge the gap between research and practical application in the healthcare domain. Since co-founding Renumics, he has been active in the field of AI for Engineering, e.g., AI for Computer Aided Engineering (CAE), implementing projects, contributing to their open-source library for data exploration for ML datasets (Renumics Spotlight) and writing articles about data visualization. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://renumics.com/ MLSecOps Community: https://community.mlsecops.com/ Blogs: https://towardsdatascience.com/visualize-your-rag-data-evaluate-your-retrieval-augmented-generation-system-with-ragas-fc2486308557 : https://medium.com/itnext/how-to-explore-and-visualize-ml-data-for-object-detection-in-images-88e074f46361 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Markus on LinkedIn: https://www.linkedin.com/in/markus-stoll-b39a42138/

MLOps for GenAI Applications // Special MLOps Podcast episode with Demetrios Brinkmann, Chief Happiness Engineer at MLOps Community. // Abstract Demetrios explores common themes in ML model testing with insights from Erica Greene (Yahoo News), Matar Haller (ActiveFence), Mohamed Elgendy (Kolena), and Catherine Nelson (Freelance Data Scientist). They discuss tiered test cases, functional testing for hate speech, differences between AI and traditional software testing, and the complexities of evaluating LLMs. Demetrios wraps up by inviting feedback and promoting an upcoming virtual conference on data engineering for AI and ML. // Bio At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps Community Podcasts. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Balancing Speed and Safety // Panel // AIQCON - https://youtu.be/c81puRgu3Kw AI For Good - Detecting Harmful Content at Scale // Matar Haller // MLOps Podcast #246 - https://youtu.be/wLKlZ6yHg1k What is AI Quality? // Mohamed Elgendy // MLOps Podcast #229 - https://youtu.be/-Jdmq4DiOew All Data Scientists Should Learn Software Engineering Principles // Catherine Nelson // Podcast #245 - https://youtu.be/yP6Eyny7p20 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Timestamps: [00:00] Exploring common themes in MLOps community [00:49] Common patterns about model output and testing [01:34] Tiered test case strategy [03:05] Functional testing for models [05:24] Testing coverage and quality [07:47] Evaluating LLMs challenges [08:35] Please like, share, leave a review, and subscribe to our MLOps channels!

Sean Morgan is an active open-source contributor and maintainer and is the special interest group lead for TensorFlow Addons. Learn more about the platform for end-to-end AI Security at https://protectai.com/. MLSecOps is Fundamental to Robust AI Security Posture Management (AISPM) // MLOps Podcast #257 with Sean Morgan, Chief Architect at Protect AI. // Abstract MLSecOps, which is the practice of integrating security practices into the AIML lifecycle (think infusing MLOps with DevSecOps practices), is a critical part of any team’s AI Security Posture Management. In this talk, we’ll discuss how to threat model realistic AIML security risks, how you can measure your organization’s AI Security Posture, and most importantly how you can improve that security posture through the use of MLSecOps. // Bio Sean Morgan is the Chief Architect at Protect AI. In prior roles he's led production AIML deployments in the semiconductor industry, evaluated adversarial machine learning defenses for DARPA research programs, and most recently scaled customers on interactive machine learning solutions at AWS. In his free time, Sean is an active open-source contributor and maintainer, and is the special interest group lead for TensorFlow Addons. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Sean's GitHub: https://github.com/seanpmorgan MLSecOps Community: https://community.mlsecops.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sean on LinkedIn: https://www.linkedin.com/in/seanmorgan/ Timestamps: [00:00] Sean's preferred coffee [00:10] Takeaways [01:39] Register for the Data Engineering for AI/ML Conference now! [02:21] KubeCon Paris: Emphasis on security and AI [05:00] Concern about malicious data during training process [09:29] Model builders, security, pulling foundational models, nuances [12:13] Hugging Face research on security issues [15:00] Inference servers exposed; potential for attack [19:45] Balancing ML and security processes for ease [23:23] Model artifact security in enterprise machine learning [25:04] Scanning models and datasets for vulnerabilities [29:23] Ray's user interface vulnerabilities lead to attacks [32:07] ML Flow vulnerabilities present significant server risks [36:04] Data ops essential for machine learning security [37:32] Prioritized security in model and data deployment [40:46] Automated scanning tool for improved antivirus protection [42:00] Wrap up

Harcharan Kabbay is a Data Scientist & AI/ML Engineer with Expertise in MLOps, Kubernetes, and DevOps, Driving End-to-End Automation and Transforming Data into Actionable Insights.

MLOps for GenAI Applications // MLOps Podcast #256 with Harcharan Kabbay, Lead Machine Learning Engineer at World Wide Technology. // Abstract The discussion begins with a brief overview of the Retrieval-Augmented Generation (RAG) framework, highlighting its significance in enhancing AI capabilities by combining retrieval mechanisms with generative models. The podcast further explores the integration of MLOps, focusing on best practices for embedding the RAG framework into a CI/CD pipeline. This includes ensuring robust monitoring, effective version control, and automated deployment processes that maintain the agility and efficiency of AI applications. A significant portion of the conversation is dedicated to the importance of automation in platform provisioning, emphasizing tools like Terraform. The discussion extends to application design, covering essential elements such as key vaults, configurations, and strategies for seamless promotion across different environments (development, testing, and production). We'll also address how to enhance the security posture of applications through network firewalls, key rotation, and other measures. Let's talk about the power of Kubernetes and related tools to aid a good application design. The podcast highlights the principles of good application design, including proper observability and eliminating single points of failure. I would share strategies to reduce development time by creating templates for GitHub repositories by application types to be re-used, also templates for pull requests, thereby minimizing human errors and streamlining the development process. // Bio Harcharan is an AI and machine learning expert with a robust background in Kubernetes, DevOps, and automation. He specializes in MLOps, facilitating the adoption of industry best practices and platform provisioning automation. With extensive experience in developing and optimizing ML and data engineering pipelines, Harcharan excels at integrating RAG-based applications into production environments. His expertise in building scalable, automated AI systems has empowered the organization to enhance decision-making and problem-solving capabilities through advanced machine-learning techniques. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Harcharan's Medium - https://medium.com/@harcharan-kabbay Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Harcharan on LinkedIn: https://www.linkedin.com/in/harcharankabbay/locale=en_US

Nicolas Mauti is an MLOps Engineer from Lyon (France), Working at Malt. BigQuery Feature Store // MLOps Podcast #255 with Nicolas Mauti, Lead MLOps at Malt. // Abstract Need a feature store for your AI/ML applications but overwhelmed by the multitude of options? Think again. In this talk, Nicolas shares how they solved this issue at Malt by leveraging the tools they already had in place. From ingestion to training, Nicolas provides insights on how to transform BigQuery into an effective feature management system. We cover how Nicolas' team designed their feature tables and addressed challenges such as monitoring, alerting, data quality, point-in-time lookups, and backfilling. If you’re looking for a simpler way to manage your features without the overhead of additional software, this talk is for you. Discover how BigQuery can handle it all! // Bio Nicolas Mauti is the go-to guy for all things related to MLOps at Malt. With a knack for turning complex problems into streamlined solutions and over a decade of experience in code, data, and ops, he is a driving force in developing and deploying machine learning models that actually work in production. When he's not busy optimizing AI workflows, you can find him sharing his knowledge at the university. Whether it's cracking a tough data challenge or cracking a joke, Nicolas knows how to keep things interesting. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Nicolas' Medium - https://medium.com/@nmauti Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Nicolas on LinkedIn: https://www.linkedin.com/in/nicolasmauti/?locale=en_US Timestamps: [00:00] Nicolas' preferred beverage [00:35] Takeaways [02:25] Please like, share, leave a review, and subscribe to our MLOps channels! [02:57] BigQuery end goal [05:00] BigQuery pain points [10:14] BigQuery vs Feature Stores [12:54] Freelancing Rate Matching issues [16:43] Post-implementation pain points [19:39] Feature Request Process [20:45] Feature Naming Consistency [23:42] Feature Usage Analysis [26:59] Anomaly detection in data [28:25] Continuous Model Retraining Process [30:26] Model misbehavior detection [33:01] Handling model latency issues [36:28] Accuracy vs The Business [38:59] BigQuery cist-benefit analysis [42:06] Feature stores cost savings [44:09] When not to use BigQuery [46:20] Real-time vs Batch Processing [49:11] Register for the Data Engineering for AI/ML Conference now! [50:14] Wrap up

Design and Development Principles for LLMOps // MLOps Podcast #254 with Andy McMahon, Director - Principal AI Engineer at Barclays Bank. A huge thank you to SAS for their generous support! // Abstract As we move from MLOps to LLMOps we need to double down on some fundamental software engineering practices, as well as augment and add to these with some new techniques. In this case, let's talk about this! // Bio Andy is a Principal AI Engineer, working in the new AI Center of Excellence at Barclays Bank. Previously he was Head of MLOps for NatWest Group, where he led their MLOps Centre of Excellence and helped build out their MLOps platform and processes across the bank. Andy is also the author of Machine Learning Engineering with Python, a hands-on technical book published by Packt. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Andy's book - https://packt.link/w3JKL Andy's Medium - https://medium.com/@andrewpmcmahon629 SAS: https://www.sas.com/en_us/home.html SAS® Decision Builder: https://www.sas.com/en_us/offers/23q4/microsoft-fabric.html Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai Harnessing MLOps in Finance // Michelle Marie Conway // MLOps Podcast Coffee #174: https://youtu.be/nIEld_Q6L-0The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses book by Eric Ries: https://www.amazon.co.jp/-/en/Eric-Ries/dp/0307887898 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Andy on LinkedIn: https://www.linkedin.com/in/andrew-p-mcmahon/

Timestamps: [00:00] Andy's preferred coffee [00:09] Takeaways [02:04] Andy's book as an Oxford curriculum [06:13] Register for the Data Engineering for AI/ML Conference now! [07:04] The Life Cycle of AI Executives Course [09:55] MLOps as a term [11:53] Tooling vs Process Culture [15:01] Open source benefits [17:15] End goal flexibility [20:06] Hybrid Cloud Strategy Overview [21:11] ROI for tool upgrades [25:41] Long-term projects comparison [29:02 - 30:48] SAS Ad [30:49] AI and ML Integration [35:40] Hybrid AI Integration Insights [42:18] Tech trends vs Practicality [44:39] Gen AI Tooling Debate [51:57] Vanity metrics overview [55:22] Tech business alignment strategy [58:45] Aligning teams for ROI [1:01:35] Communication mission effectively [1:03:45] Enablement metrics [1:06:38] Prioritizing use cases [1:09:47] Wrap up

// Abstract Data is the foundation of AI. To ensure AI performs as expected, high-quality data is essential. In this panel discussion, Chad, Maria, Joe, and Pushkar hosted by Sam Partee will explore strategies for obtaining and maintaining high-quality data, as well as common pitfalls to avoid when using data for AI models. // Panelists - Samuel Partee: Principal Applied AI Engineer @ Redis - Chad Sanderson: CEO & Co-Founder @ Gable - Joe Reis: CEO/Co-Founder @ Ternary Data - Maria Zhang: CEO Cofounder @ Proactive AI Lab Inc - Pushkar Garg: Staff Machine Learning Engineer @ Clari Inc.

Yuri Plotkin is a Biomedical Engineer and Machine Learning Scientist and the author of The Variational Book.

The Variational Book // MLOps Podcast #253 with Yuri Plotkin, an ML Scientist. // Abstract Curiosity has been the underlying thread in Yuri's life and interests. With the explosion of Generative AI, Yuri was fascinated by the topic and decided he needed to learn more. Yuri pursued learning by reading, deriving, and understanding seminal papers within the last generation. The endeavors culminated in the writing of a book on the topic, The Variational Book, which Yuri expects to release shortly in the coming months. A bit of detail about the topics he covers can be found here: www.thevariationalbook.com. // Bio Evolved from biomedical engineer to wet-lab scientist, and more recently transitioned Yuri's career to computer science with the last 10+ years developing projects at the intersection of medicine, life sciences and machine learning. Yuri's educational background is in Biomedical Engineering, at Columbia University (M.S.) and University of California, San Diego (B.S.). Current interests include generative AI, diffusion models, and LLMs. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://plotkiny.github.io/ The Variational Book: www.thevariationalbook.com SAS: https://www.sas.com/en_us/home.html SAS® Decision Builder: https://www.sas.com/en_us/offers/23q4/microsoft-fabric.html

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Yuri on LinkedIn: http://www.linkedin.com/in/yuri-plotkin/

// Abstract Attracting and retaining top AI talent is essential for staying competitive. This panel will explore crafting and communicating a compelling vision that aligns with the organization's evolving needs, inspiring potential hires and motivating current employees. The discussion will offer actionable strategies for sourcing top talent, adapting to changing needs, and maintaining company alignment. Attendees will learn best practices for attracting AI professionals, creating an attractive employer brand, and enhancing talent acquisition and retention strategies. Lastly, the panel will cover structuring and organizing the AI team as it grows to ensure alignment with business goals. This includes optimal team configurations, leadership roles, and processes that support collaboration and innovation, enabling sustained growth and success. // PANELISTS Ashley Antonides: Associate Research Director, AI/ML @ Two Six Technologies Olga Beregovaya: VP, AI @ Smartling Shailvi Wakhlu: Founder @ Shailvi Ventures LLC A big thank you to our Premium Sponsors Google Cloud & Databricks for their generous support!

Ron Heichmn is an AI researcher specializing in generative AI, AI alignment, and prompt engineering. At SentinelOne, Ron actively monitors emerging research to identify and address potential vulnerabilities in our AI systems, focusing on unsupervised and scalable evaluations to ensure robustness and reliability.

Harnessing AI APIs for Safer, Accurate, & Reliable Applications // MLOps Podcast #252 with Ron Heichman, Machine Learning Engineer at SentinelOne. // Abstract Integrating AI APIs effectively is pivotal for building applications that leverage LLMs, especially given the inherent issues with accuracy, reliability, and safety that LLMs often exhibit. I aim to share practical strategies and experiences for using AI APIs in production settings, detailing how to adapt these APIs to specific use cases, mitigate potential risks, and enhance performance. The focus will be testing, measuring, and improving quality for RAG or knowledge workers utilizing AI APIs. // Bio Ron Heichman is an AI researcher and engineer dedicated to advancing the field through his work on prompt injection at Preamble, where he helped uncover critical vulnerabilities in AI systems. Currently at SentinelOne, he specializes in generative AI, AI alignment, and the benchmarking and measurement of AI system performance, focusing on Retrieval-Augmented Generation (RAG) and AI guardrails. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.sentinelone.com/

All the Hard Stuff with LLMs in Product Development // Phillip Carter // MLOps Podcast #170: https://www.youtube.com/watch?v=DZgXln3v85s&ab_channel=MLOps.community --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ron on LinkedIn: https://www.linkedin.com/in/heichmanron/

This is a panel taken from the recent AI quality Conference presented by the MLOps Community and Kolena

// Abstract The need for moving to production quickly is paramount in staying out of perpetual POC territory. AI is moving fast. Shipping features fast to stay ahead of the competition is commonplace. Quick iterations are viewed as strength in the startup ecosystem, especially when taking on a deeply entrenched competitor. Each week a new method to improve your AI system becomes popular or a SOTA foundation model is released. How do we balance the need for speed vs the responsibility of safety? Having the confidence to ship a cutting-edge model or AI architecture and knowing it will perform as tasked. What are the risks and safety metrics that others are using when they deploy their AI systems. How can you correctly identify when risks are too large? // Panelists - Remy Thellier: Head of Growth & Strategic Partnerships @ Vectice - Erica Greene: Director of Engineering, Machine Learning @ Yahoo - Shreya Rajpal: Creator @ Guardrails AI A big thank you to our Premium Sponsors Google Cloud & Databricks for their generous support!

Chinar Movsisyan is the co-founder and CEO of Feedback Intelligence (formerly Manot), an MLOps startup based in San Francisco. She has been in the AI field for more than 7 years from research labs to venture-backed startups.

Reliable LLM Products, Fueled by Feedback // MLOps Podcast #250 with Chinar Movsisyan, CEO of Feedback Intelligence. // Abstract We live in a world driven by large language models (LLMs) and generative AI, but ensuring they are ready for real-world deployment is crucial. Despite the availability of numerous evaluation tools, many LLM products still struggle to make it to production. We propose a new perspective on how LLM products should be measured, evaluated, and improved. A product is only as good as the user's experience and expectations, and we aim to enhance LLM products to meet these standards reliably. Our approach creates a new category that automates the need for separate evaluation, observability, monitoring, and experimentation tools. By starting with the user experience and working backward to the model, we provide a comprehensive view of how the product is actually used, rather than how it is intended to be used. This user-centric aka feedback-centric approach is the key to every successful product. // Bio Chinar Movsisyan is the founder and CEO of Feedback Intelligence, an MLOps company based in San Francisco that enables enterprises to make sure that LLM-based products are reliable and that the output is aligned with end-user expectations. With over eight years of experience in deep learning, spanning from research labs to venture-backed startups, Chinar has led AI projects in mission-critical applications such as healthcare, drones, and satellites. Her primary research interests include artificial intelligence, generative AI, machine learning, deep learning, and computer vision. At Feedback Intelligence, Chinar and her team address a crucial challenge in LLM development by automatically converting user feedback into actionable insights, enabling AI teams to analyze root causes, prioritize issues, and accelerate product optimization. This approach is particularly valuable in highly regulated industries, helping enterprises to reduce time-to-market and time-to-resolution while ensuring robust LLM products. Feedback Intelligence, which participated in the Berkeley SkyDeck accelerator program, is currently expanding its business across various verticals. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.manot.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Chinar on LinkedIn: https://www.linkedin.com/in/nik-suresh/ Timestamps: [00:00] Chinar's preferred coffee [00:20] Takeaways [02:25] Please like, share, leave a review, and subscribe to our MLOps channels! [03:23] Object Detection on Drones [06:10] Street Surveillance Detection Use Case [08:00] Optimizing Vision Models [09:50] Data Engineering for AI/ML Conference Ad [10:42] Plastic surgery project [12:33] Diffusion models getting popular [13:57] AI challenges in highly regulated industries [17:48] Product metrics evaluation insights [20:55] Chatbot effectiveness metrics [23:15] Interpreting user signals [24:45] Metadata tracking in LLM [27:41] Agentic workflow [28:53] Effective data analysis strategies [30:41] Identifying key metrics [33:59] AI metrics role shift [37:20] Tooling for non-engineers [42:12] Balancing engineering and evaluation [44:39] Bridging SME engineering gap [46:41] Expand expertise potential [47:40] What's with flamingos [48:04] Wrap up

This is a Panel taken from the recent AI Quality Conference presented by the MLOps COmmunity and Kolena

// Abstract Enterprise AI leaders continue to explore the best productivity solutions that solve business problems, mitigate risks, and increase efficiency. Building reliable and secure AI/ML systems requires following industry standards, an operating framework, and best practices that can accelerate and streamline the scalable architecture that can produce expected business outcomes. This session, featuring veteran practitioners, focuses on building scalable, reliable, and quality AI and ML systems for the enterprises. // Panelists - Hira Dangol: VP, AI/ML and Automation @ Bank of America - Rama Akkiraju: VP, Enterprise AI/ML @ NVIDIA - Nitin Aggarwal: Head of AI Services @ Google - Steven Eliuk: VP, AI and Governance @ IBM A big thank you to our Premium Sponsors Google Cloud & Databricks for their generous support!

Timestamps:

00:00 Panelists discuss vision and strategy in AI

05:18 Steven Eliuk, IBM expertise in data services

07:30 AI as means to improve business metrics

11:10 Key metrics in production systems: efficiency and revenue

13:50 Consistency in data standards aids data integration

17:47 Generative AI presents new data classification risks

22:47 Evaluating implications, monitoring, and validating use cases

26:41 Evaluating natural language answers for efficient production

29:10 Monitoring AI models for performance and ethics

31:14 AI metrics and user responsibility for future models

34:56 Access to data is improving, promising progress

Nik Suresh wrote an evisceration of the current AI hype boom called "I Will F**king Piledrive You If You Mention AI Again." AI Operations Without Fundamental Engineering Discipline // MLOps Podcast #250 with Nikhil Suresh, Director @ Hermit Tech. // Abstract Nik is on the podcast because of an anti-AI hype piece, so a reasonable thing to discuss is going to be what most companies are getting wrong when non-technical management wants to immediately roll out ML initiatives, but are unwilling to bring technical naysayers on board who will set them up for success. // Bio Nik is the author of ludic.mataroa.blog, who wrote "I Will [REDACTED] Piledriver You If You Mention AI Again", and mostly works in the data engineering and data science spaces. Nik's writing and company both focus on bringing more care to work, pushing back against the industry's worst excesses both technically and spiritually, and getting fundamentals right. Nik also has a reasonably strong background in psychology. His data science training was of the pre-LLM variety, circa. 2018, when there was a lot of hype but it wasn't this ridiculous. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://ludic.mataroa.blog/

Nik's blog: https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you-if-you-mention-ai-again/ Harnessing MLOps in Finance // Michelle Marie Conway // MLOps Podcast Coffee #174: https://youtu.be/nIEld_Q6L-0

Fundamentals of Data Engineering: Plan and Build Robust Data Systems AudiobookBy: Joe Reis, Matt Housley: https://audiobookstore.com/audiobooks/fundamentals-of-data-engineering.aspx

Bullshit Jobs A Theory Hardcover by David Graeber: https://www.amazon.co.jp/-/en/David-Graeber/dp/0241263883 Does a Frog have Scorpion Nature podcast: https://open.spotify.com/show/57i8sYVqxG4i3NvBniLfhv --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Nik on LinkedIn: https://www.linkedin.com/in/nik-suresh/

Timestamps: [00:00] Nik's preferred coffee [00:30] Takeaways [01:40] Please like, share, leave a review, and subscribe to our MLOps channels! [01:56] AI hype and humor [07:21] Defining project success [08:57] Effective data utilization [12:18] AI Hype vs Data Engineering [14:44] AI implementation challenges [17:44 - 18:35] Data Engineering for AI and ML Virtual Conference Ad [18:35] Managing AI Expectations [22:08] AI expectations vs reality [26:00] Balancing Engineering and AI [31:54] Highlighting engineer success [35:25] The real challenges [36:30] Embracing work challenges [37:21] Dealing with podcast disappointments [40:50] Creating content for visibility [43:02] Exploring niche interests [44:14] Relationship building [47:15] Strategic approach to success [48:36] Wrap up

Eric Landry is a seasoned AI and Machine Learning leader with extensive expertise in software engineering and practical applications in NLP, document classification, and conversational AI. With technical proficiency in Java, Python, and key ML tools, he leads the Expedia Machine Learning Engineering Guild and has spoken at major conferences like Applied Intelligence 2023 and KDD 2020. AI in Healthcare // MLOps Podcast #249 with Eric Landry, CTO/CAIO @ Zeteo Health. // Abstract Eric Landry discusses the integration of AI in healthcare, highlighting use cases like patient engagement through chatbots and managing medical data. He addresses benchmarking and limiting hallucinations in LLMs, emphasizing privacy concerns and data localization. Landry maintains a hands-on approach to developing AI solutions and navigating the complexities of healthcare innovation. Despite necessary constraints, he underscores the potential for AI to proactively engage patients and improve health outcomes. // Bio Eric Landry is a technology veteran with 25+ years of experience in the healthcare, travel, and computer industries, specializing in machine learning engineering and AI-based solutions. Holding a Masters in SWE (NLP thesis topic) from the University of Texas at Austin, 2005. He has showcased his expertise and leadership in the field with three US patents, published articles on machine learning engineering, and speaking engagements at the 2023 Applied Intelligence Live, 2020 KDD conference, Data Science Salon 2024, and former leader of Expedia’s MLE guild. Formerly, Eric was the director of AI Engineering and Conversation Platform at Babylon Health and Expedia. Currently CTO/CAIO at Zeteo Health. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.zeteo.health/ Building Threat Detection Systems: An MLE's Perspective // Jeremy Jordan // MLOps Podcast #134: https://youtu.be/13nOmMJuiAo --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Eric on LinkedIn: https://www.linkedin.com/in/jeric-landry/ Timestamps: [00:00] Eric's preferred coffee [00:16] Takeaways [01:16] Please like, share, leave a review, and subscribe to our MLOps channels! [01:32] ML and AI in 2005 [04:43] Last job at Babylon Health [10:57] Data access solutions [14:35] Prioritize AI ML Team Success [16:39] Eric's current work [20:36] Engage in holistic help [22:13] High-stakes chatbots [27:30] Navigating Communication Across Diverse Communities [31:49] When Bots Go Wrong [34:15] Health care challenges ahead [36:05] Behavioral health tech challenges [39:45] Stress from Apps Notifications [41:11] Combining different guardrails tools [47:16] Navigating Privacy AI [50:12] Wrap up

Aniket Kumar Singh is a Vision Systems Engineer at Ultium Cells, skilled in Machine Learning and Deep Learning. I'm also engaged in AI research, focusing on Large Language Models (LLMs). Evaluating the Effectiveness of Large Language Models: Challenges and Insights // MLOps Podcast #248 with Aniket Kumar Singh, CTO @ MyEvaluationPal | ML Engineer @ Ultium Cells. // Abstract Dive into the world of Large Language Models (LLMs) like GPT-4. Why is it crucial to evaluate these models, how we measure their performance, and the common hurdles we face? Drawing from Aniket's research, he shares insights on the importance of prompt engineering and model selection. Aniket also discusses real-world applications in healthcare, economics, and education, and highlights future directions for improving LLMs. // Bio Aniket is a Vision Systems Engineer at Ultium Cells, skilled in Machine Learning and Deep Learning. I'm also engaged in AI research, focusing on Large Language Models (LLMs). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: www.aniketsingh.me Aniket's AI Research for Good blog that I plan to utilize to share any new research that would focus on the good: www.airesearchforgood.org Aniket's papers: https://scholar.google.com/citations?user=XHxdWUMAAAAJ&hl=en --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Aniket on LinkedIn: https://www.linkedin.com/in/singh-k-aniket/ Timestamps: [00:00] Aniket's preferred coffee [00:14] Takeaways [01:29] Aniket's job and hobby [03:06] Evaluating LLMs: Systems-Level Perspective [05:55] Rule-based system [08:32] Evaluation Focus: Model Capabilities [13:04] LLM Confidence [13:56] Problems with LLM Ratings [17:17] Understanding AI Confidence Trends [18:28] Aniket's papers [20:40] Testing AI Awareness [24:36] Agent Architectures Overview [27:05] Leveraging LLMs for tasks [29:53] Closed systems in Decision-Making [31:28] Navigating model Agnosticism [33:47] Robust Pipeline vs Robust Prompt [34:40] Wrap up

Sophia Rowland is a Senior Product Manager focusing on ModelOps and MLOps at SAS. In her previous role as a data scientist, Sophia worked with dozens of organizations to solve a variety of problems using analytics.

David Weik has a passion for data and creating integrated customer-centric solutions. Thinking data and people first to create value-added solutions. Extending AI: From Industry to Innovation // MLOps Podcast #247 with Sophia Rowland, Senior Product Manager and David Weik, Senior Solutions Architect of SAS. Huge thank you to SAS for sponsoring this episode. SAS - http://www.sas.com/ // Abstract Organizations worldwide invest hundreds of billions into AI, but they do not see a return on their investments until they are able to leverage their analytical assets and models to make better decisions. At SAS, we focus on optimizing every step of the Data and AI lifecycle to get high-performing models into a form and location where they drive analytically driven decisions. Join experts from SAS as they share learnings and best practices from implementing MLOps and LLMOPs at organizations across industries, around the globe, and using various types of models and deployments, from IoT CV problems to composite flows that feature LLMs. // Bio Sophia Rowland Sophia Rowland is a Senior Product Manager focusing on ModelOps and MLOps at SAS. In her previous role as a data scientist, Sophia worked with dozens of organizations to solve a variety of problems using analytics. As an active speaker and writer, Sophia has spoken at events like All Things Open, SAS Explore, and SAS Innovate as well as written dozens of blogs and articles. As a staunch North Carolinian, Sophia holds degrees from both UNC-Chapel Hill and Duke including bachelor’s degrees in computer science and psychology and a Master of Science in Quantitative Management: Business Analytics from the Fuqua School of Business. Outside of work, Sophia enjoys reading an eclectic assortment of books, hiking throughout North Carolina, and trying to stay upright while ice skating. David Weik David joined SAS in 2020 as a solutions architect. He helps customers to define and implement data-driven solutions. Previously, David was a SAS administrator/developer at a German insurance company working with the integration capabilities of SAS, Robotic Process Automation, and more. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links http://www.sas.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sophia on LinkedIn: https://www.linkedin.com/in/sophia-rowland/ Connect with David on LinkedIn: https://www.linkedin.com/in/david-weik/ Timestamps: [00:00] Sophia & David's preferred coffee [00:19] Takeaways [02:11] Please like, share, leave a review, and subscribe to our MLOps channels! [02:55] Hands on MLOps and AI [05:14] Next-Gen MLOps Challenges [07:24] Data scientists adopting software [11:48] Taking a different approach [13:43] Zombie Model Management [16:36] Optimizing ML Revenue Allocation [18:39] Other use cases - Lockout - Tagout procedure [21:43] Vision Model Integration Challenges [26:16] Costly errors in predictive maintenance [27:25] Integration of Gen AI [34:32] Governance challenges in AI [38:00] Governance in Gen AI vs Governance with Traditional ML [41:53] Evaluation challenges in industries [46:49] Interface frustration with Chatbots [51:25] Implementing AI Agent's success [54:18] Usability challenges in interfaces [57:03] Themes in High-Performing AI Teams [1:00:51] Wrap up

Matar Haller is the VP of Data & AI at ActiveFence, where her teams own the end-to-end automated detection of harmful content at scale, regardless of the abuse area or media type. The work they do here is engaging, impactful, and tough, and Matar is grateful for the people she gets to do it with.

AI For Good - Detecting Harmful Content at Scale // MLOps Podcast #246 with Matar Haller, VP of Data & AI at ActiveFence. // Abstract One of the biggest challenges facing online platforms today is detecting harmful content and malicious behavior. Platform abuse poses brand and legal risks, harms the user experience, and often represents a blurred line between online and offline harm. So how can online platforms tackle abuse in a world where bad actors are continuously changing their tactics and developing new ways to avoid detection? // Bio Matar Haller leads the Data & AI Group at ActiveFence, where her teams are responsible for the data, algorithms, and infrastructure that fuel ActiveFence’s ability to ingest, detect, and analyze harmful activity and malicious content at scale in an ever-changing, complex online landscape. Matar holds a Ph.D. in Neuroscience from the University of California at Berkeley, where she recorded and analyzed signals from electrodes surgically implanted in human brains. Matar is passionate about expanding leadership opportunities for women in STEM fields and has three children who surprise and inspire her every day. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links activefence.comhttps://www.youtube.com/@ActiveFence --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Matar on LinkedIn: https://www.linkedin.com/company/11682234/admin/feed/posts/ Timestamps: [00:00] Matar's preferred coffee [00:13] Takeaways [01:39] The talk that stood out [06:15] Online hate speech challenges [08:13] Evaluate harmful media API [09:58] Content moderation: AI models [11:36] Optimizing speed and accuracy [13:36] Cultural reference AI training [15:55] Functional Tests [20:05] Continuous adaptation of AI [26:43] AI detection concerns [29:12] Fine-Tuned vs Off-the-Shelf [32:04] Monitoring Transformer Model Hallucinations [34:08] Auditing process ensures accuracy [38:38] Testing strategies for ML [40:05] Modeling hate speech deployment [42:19] Improving production code quality [43:52] Finding balance in Moderation [47:23] Model's expertise: Cultural Sensitivity [50:26] Wrap up

Catherine Nelson is a freelance data scientist and writer. She is currently working on the forthcoming O’Reilly book "Software Engineering for Data Scientists”. Why All Data Scientists Should Learn Software Engineering Principles // MLOps podcast #245 with Catherine Nelson, a freelance Data Scientist. A big thank you to LatticeFlow AI for sponsoring this episode! LatticeFlow AI - https://latticeflow.ai/ // Abstract Data scientists have a reputation for writing bad code. This quote from Reddit sums up how many people feel: “It's honestly unbelievable and frustrating how many Data Scientists suck at writing good code.” But as data science projects grow, and because the job now often includes deploying ML models, it's increasingly important for DSs to learn fundamental SWE principles such as keeping your code modular, making sure your code is readable by other people and so on. The exploratory nature of DS projects means that you can't be sure where you will end up at the start of a project, but there's still a lot you can do to standardize the code you write. // Bio Catherine Nelson is the author of "Software Engineering for Data Scientists", a guide for data scientists who want to level up their coding skills, published by O'Reilly in May 2024. She is currently consulting for GenAI startups and providing mentorship and career coaching to data scientists. Previously, she was a Principal Data Scientist at SAP Concur. She has extensive experience deploying NLP models to production and evaluating ML systems, and she is also co-author of the book "Building Machine Learning Pipelines", published by O'Reilly in 2020. In her previous career as a geophysicist, she studied ancient volcanoes and explored for oil in Greenland. Catherine has a PhD in geophysics from Durham University and a Masters of Earth Sciences from Oxford University. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Software Engineering for Data Scientists book by Catherine Nelson: https://learning.oreilly.com/library/view/software-engineering-for/9781098136192/ https://www.amazon.com/Software-Engineering-Data-Scientists-Notebooks/dp/1098136209 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Catherine on LinkedIn: https://www.linkedin.com/in/catherinenelson1/ Timestamps: [00:00] Catherine's preferred coffee [00:15] Takeaways [02:38] Meeting magic: Embracing serenity [06:23] The Software Engineering for Data Scientists book [10:41] Exploring ideas rapidly [12:52] Bridging Data Science gaps [16:17] Data poisoning concerns [18:26] Transitioning from a data scientist to a machine learning engineer [21:53] Rapid Prototyping vs Thorough Development [23:45] Data scientists take ownership [25:53] Data scientists' role balance [30:30] Understanding system design process [36:00] Data scientists and Kubernetes [41:33 - 43:03] LatticeFlow AI Ad [43:05] The Future of Data Science [45:09] Data scientists analyzing models [46:46] Tools gaps in prompt tracking [50:44] Learnings from writing the book

Meta GenAI Infra Blog Review // Special MLOps Podcast episode by Demetrios. // Abstract Demetrios explores Meta's innovative infrastructure for large-scale AI operations, highlighting three blog posts on training large language models, maintaining AI capacity, and building Meta's GenAI infrastructure. The discussion reveals Meta's handling of hundreds of trillions of AI model executions daily, focusing on scalability, cost efficiency, and robust networking. Key elements include the Ops planner work orchestrator, safety protocols, and checkpointing challenges in AI training. Meta's efforts in hardware design, software solutions, and networking optimize GPU performance, with innovations like a custom Linux file system and advanced networking file systems like Hammerspace. The podcast also discusses advancements in PyTorch, network technologies like Roce and Nvidia's Quantum 2 Infiniband fabric, and Meta's commitment to open-source AGI. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Building Meta’s GenAI Infrastructure blog: https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Timestamps:

[00:00] Meta handles trillions of AI model executions

[07:01] Meta creating AGI, ethical and sustainable

[08:13] Concerns about energy use in training models

[12:22] Network, hardware, and job optimization for reliability

[17:21] Highlights of Arista and Nvidia hardware architecture

[20:11] Meta's clusters optimized for efficient fabric

[24:40] Varied steps, careful checkpointing in AI training

[28:46] Meta is maintaining huge GPU clusters for AI

[29:47] AI training is faster and more demanding

[35:27] Ops planner orchestrates a million operations and reduces maintenance

[37:15] Ops planner ensures safety and well-tested changes

Sean Wei, the CEO and co-founder of RealChar, shares his journey from working in the autonomous vehicle industry to creating an open-source voice assistant project called Realchar, which eventually evolved into Rivia, a voice AI assistant focused on managing personal phone calls. The Future of AI and Consumer Empowerment // MLOps podcast #244 with Shaun Wei, CEO & Co-Founder of RealChar. A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/ // Abstract Explore the groundbreaking work RealChar is doing with its consumer application, Rivia. This discussion focuses on how Rivia leverages Generative AI and Traditional Machine Learning to handle mundane phone calls and customer service interactions, aiming to free up human time for more meaningful tasks. The product, currently in beta, embodies a forward-thinking approach to AI, where the technology offloads day-to-day burdens like scheduling appointments and making calls. // Bio Shaun Wei is a well-connected technology professional with a rich background in developing and analyzing artificial intelligence systems. In 2018, Shaun played a pivotal role in the advent and deployment of Google Duplex, a remarkable AI capable of handling natural conversations and performing tasks such as booking hair salon appointments and restaurant reservations via telephone. His involvement wasn't just limited to the developmental side; Shaun also uniquely positioned himself on the receiving end, gathering insights by interviewing users directly impacted by the technology. This dual perspective has enabled Shaun to grasp both the technical underpinnings and the human-centric applications of AI, making him a valuable asset in the tech industry. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.rivia.tech/

https://realchar.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Shaun on LinkedIn: https://www.linkedin.com/in/shaunwei/

Timestamps: [00:00] Shaun's preferred coffee [00:28] Takeaways [03:30] Please like, share, leave a review, and subscribe to our MLOps channels! [03:57] AI in Production: Challenges & Insights [06:13] AI Scheduling and Assistance [08:00] Technical Challenges in AI [12:36 - 14:06] LatticeFlow Ad [14:09] Handling Challenges in AI [15:52] Learning driving and technical aspects [19:04] Self-Driving Cars: Multimodal Integration [23:41] Processing data with Transformers [26:46] Real-time phone data gathering [30:49] Real-time observability in AI [35:09] Time to first token [37:26] Preferred vs. Dynamic Model Selection [40:12] Event-driven architecture basics [42:06] Navigating challenges together [44:02] Challenges with Inconsistent Responses [45:40] Importance of product reliability [47:47] Training Data and Model Performance [50:02] Exploring AI in Customer Service [51:34] Navigating challenges in AI [53:15] Excited Launch Strategy Advice [57:10] Wrap up

Join us at our first in-person conference today all about AI Quality: https://www.aiqualityconference.com/ ML and AI as Distinct Control Systems in Heavy Industrial Settings // MLOps podcast #243 with Richard Howes, CTO of Metaformed.

Richard Howes is a dedicated engineer who is passionate about control systems whether it be embedded systems, industrial automation, or AI/ML in a business application. Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/ // Abstract How can we balance the need for safety, reliability, and robustness with the extreme pace of technology advancement in heavy industry? The key to unlocking the full potential of data will be to have a mixture of experts both from an AI and human perspective to validate anything from a simple KPI to a Generative AI Assistant guiding operators throughout their day. The data generated by heavy industries like agriculture, oil & gas, forestry, real estate, civil infrastructure, and manufacturing is underutilized and struggles to keep up with the latest and greatest - and for good reason. They provide the shelter we live and work in, the food we eat, and the energy to propel society forward. Compared to the pace of AI innovation they move slowly, have extreme consequences for failure, and typically involve a significant workforce. During this discussion, we will outline the data ready to be utilized by ML, AI, and data products in general as well as some considerations for creating new data products for these heavy industries. To account for complexity and uniqueness throughout the organization it is critical to engage operational staff, ensure safety is considered from all angles, and build adaptable ETL needed to bring the data to a usable state. // Bio Richard Howes is a dedicated engineer who is passionate about control systems whether it be embedded systems, industrial automation, or AI/ML in a business application. All of these systems require a robust control philosophy that outlines the system, its environment, and how the controller should function within it. Richard has a bachelor's of Electrical Engineering from the University of Victoria where he specialized in industrial automation and embedded systems. Richard is primarily focused on the heavy industrial sectors like energy generation, oil & gas, pulp/paper, forestry, real estate, and manufacturing. He works on both physical process control and business process optimization using the control philosophy principles as a guiding star. Richard has been working with industrial systems for over 10 years designing, commissioning, operating, and maintaining automated systems. For the last 5 years, Richard has been investing time into the data and data science-related disciplines bringing the physical process as close as possible to the business taking advantage of disparate data sets throughout the organization. Now with the age of AI upon us, he is focusing on integrating this technology safely, reliably, and with distinct organizational goals and ROI. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AWS Trainium: https://aws.amazon.com/machine-learning/trainium/ AWS Inferentia: https://aws.amazon.com/machine-learning/inferentia/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Richard on LinkedIn: https://www.linkedin.com/in/richardhowes/

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ Accelerating Multimodal AI // MLOps podcast #241 with Ethan Rosenthal, Member of Technical Staff of Runway. Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/ // Abstract We’re still trying to figure out systems and processes for training and serving “regular” machine learning models, and now we have multimodal AI to contend with! These new systems present unique challenges across the spectrum, from data management to efficient inference. I’ll talk about the similarities, differences, and challenges that I’ve seen by moving from tabular machine learning, to large language models, to generative video systems. I’ll also talk about the setups and tools that I have seen work best for supporting and accelerating both the research and productionization process. // Bio Ethan works at Runway building systems for media generation. Ethan's work generally straddles the boundary between research and engineering without falling too hard on either side. Prior to Runway, Ethan spent 4 years at Square. There, he led a small team of AI Engineers training large language models for Conversational AI. Before Square, Ethan freelance consulted and worked at a couple ecommerce startups. Ethan found his way into tech by way of a Physics PhD. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.ethanrosenthal.com Ethan's mangum opus: https://www.ethanrosenthal.com/2020/08/25/optimal-peanut-butter-and-banana-sandwiches/ Real-time Model Inference in a Video Streaming Environment // Brannon Dorsey // Coffee Sessions #98: https://youtu.be/TNO6rYwP3yg Feature Stores for Self-Service Machine Learning: https://www.ethanrosenthal.com/2021/02/03/feature-stores-self-service/ Gen-1: The Next Step Forward for Generative AI: https://research.runwayml.com/gen1 Machine Learning: The High Interest Credit Card of Technical Debt by D. Sculley et al.: https://research.google/pubs/machine-learning-the-high-interest-credit-card-of-technical-debt/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ethan on LinkedIn: https://bsky.app/profile/ethanrosenthal.com

Timestamps: [00:00] Ethan's preferred coffee [00:11] Takeaways [02:07] Falling into LLMs [03:16] Advanced AI Tech Capabilities [04:40] AI-powered video editing tool [06:56] Transition to AI: Diffusion Models [09:09] Multimodal Feature Store breakdown [15:33] Multimodal Feature Stores Evolution [18:09] Benefits of Multimodal Feature Store [25:09] Centralized Training Data Repository [27:33] Large-scale distributed training [32:37 - 33:39] AWS Ad [33:45] Dealing with researchers on productionizing [43:52] Infrastructure for Researchers and Engineers [47:04] Generative DevOps movement [49:21] Structuring teams [52:06] Multimodal Feature Stores Efficiency [54:02] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ Navigating the AI Frontier: The Power of Synthetic Data and Agent Evaluations in LLM Development // MLOps podcast #241 with Boris Selitser, Co-Founder and CTO/CPO of Okareo. A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/ // Abstract Explore the evolving landscape of building LLM applications, focusing on the critical roles of synthetic data and agent evaluations. Discover how synthetic data enhances model behavior description, prototyping, testing, and fine-tuning, driving robustness in LLM applications. Learn about the latest methods for evaluating complex agent-based systems, including RAG-based evaluations, dialog-level assessments, simulated user interactions, and adversarial models. This talk delves into the specific challenges developers face and the tradeoffs involved in each evaluation approach, providing practical insights for effective AI development. // Bio Boris is the Co-Founder and CTO/CPO at Okareo. Okareo is a full-cycle platform for developers to evaluate and customize AI/LLM applications. Before Okareo, Boris was Director of Product at Meta/Facebook, leading teams building internal platforms and ML products. Examples include a copyright classification system across the Facebook apps and an engagement platform for over 200K developers, 500K+ creators, and 12M+ Oculus users. Boris has a bachelor’s in Computer Science from UC Berkeley. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://docs.okareo.com/blog/data_loop https://docs.okareo.com/blog/agent_eval The Real E2E RAG Stack // Sam Bean // MLOps Podcast #217 - https://youtu.be/8uZst7pgOw0

RecSys at Spotify // Sanket Gupta // MLOps Podcast #232 - https://youtu.be/byH-ARJA4gk --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Boris on LinkedIn: https://www.linkedin.com/in/selitser/

Timestamps: [00:00] Boris' preferred coffee [00:37] Takeaways [02:32] Please like, share, leave a review, and subscribe to our MLOps channels! [02:48] Software Engineering and Data Science [06:01] AI Transformative Potential Explained [10:31] Prompt Injection Protection Strategies [17:03] Agent's metrics for Jira [24:11] Data and Metrics Evolution [27:54] Evaluation Focus Enhances Systems [31:22 - 32:52] LatticeFlow AD [32:55] Custom Evaluation and Synthetic Data [36:23] Synthetic data for expansion, evaluation, and map [41:06] Diverse agents' personalities for readiness [44:25] Agent functions [46:17] Optimizing Routing Agents [50:04] Adapting to tool output for decision-making [52:56] Agent framework evolution [55:41] Agent framework for delivering value [57:03] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ MLOps Coffee Sessions Special episode with LatticeFlow, How to Build Production-Ready AI Models for Manufacturing, fueled by our Premium Brand Partner, LatticeFlow. Deploying AI models in manufacturing involves navigating several technical challenges such as costly data acquisition, class imbalances, data shifts, leakage, and model degradation over time. How can you uncover the causes of model failures and prevent them effectively? This discussion covers practical solutions and advanced techniques to build resilient, safe, and high-performing AI systems in the manufacturing industry. // Bio Pavol Bielik Pavol earned his PhD at ETH Zurich, specializing in machine learning, symbolic AI, synthesis, and programming languages. His groundbreaking research earned him the prestigious Facebook Fellowship in 2017, representing the sole European recipient, along with the Romberg Grant in 2016. Following his doctorate, Pavol's passion for ensuring the safety and reliability of deep learning models led to the founding of LatticeFlow. Building on a more than a decade of research, Pavol and a dynamic team of researchers at LatticeFlow developed a platform that equips companies with the tools to deliver robust and high-performance AI models, utilizing automatic diagnosis and improvement of data and models. Aniket Singh Vision Systems Engineer AI Researcher Mohan Mahadevan Mohan Mahadevan is a seasoned technology leader with 25 years of experience in building computer vision (CV) and machine learning (ML) based products. Mohan has led teams to successfully deliver real world solutions spanning hardware, software, and AI based solutions in over 20 product families across a diverse range of domains, including Semiconductors, Robotics, Fintech, and Insuretech. Mohan Mahadevan has led global teams in the development of cutting-edge technologies across a range of disciplines including computer vision, machine learning, optical and hardware architectures, system design, computational optimization and more. Jürgen Weichenberger 20+ years of advanced analytics, data science, database design, architecture, and implementation on various platforms to solve Complex Industry Problems. Industrial Analytics is the fusion of manufacturing, production, reliability, integrity, quality, sales- and market-analytics and covering 10 Industries. By combining skills and experience, we are creating the next-generation AI & ML Solutions for our clients. Leveraging a unique formula which allows us to model some of the most challenging manufacturing problems while building, scaling, and enabling the end-user to leverage the next generation data products. The Strategy & Innoation Team at Schneider is specialising on Industrial-Grade Challenges where we are applying ML & AI methods to achieve state of the art results. Personally, I am driving my team and my own education to extend the limits of AI & ML beyond the current possible. I hold more than 15 patents and I am working on new innovations. I am working with our partner eco-system to enrich our accelerators with modern ML/AI techniques and integrating robotic equipment allows me to create next generation solutions. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://latticeflow.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Timestamps: [00:00] Demetrios' Intro [00:48] Announcements [01:57] Join us at our first in-person conference on June 25 all about AI Quality! [03:39] Speakers' intros [06:00] AI ML uncommon use cases [10:14] Challenges in Implementing AI and ML in Heavy Industries [11:41] Optimizing AI use cases [18:07] Moving from PoC to Production [20:53] Hybrid AI Integration for Safety [28:28] Training AI for Defect Variability [33:18] Challenges in AI Integration [35:39] Metrics for Evaluating Success [37:27] Challenges in AI Integration [44:39] Usage of LLMs [50:34] Fine-tuning AI Models [53:20] Trust Dynamics: TML vs LLM [55:23] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Miguel Fierro is a Principal Data Science Manager at Microsoft and holds a PhD in robotics.

From Robotics to Recommender Systems // MLOps Podcast #240 with Miguel Fierro, Principal Data Science Manager at Microsoft. Huge thank you to Zilliz for sponsoring this episode. Zilliz - https://zilliz.com/. // Abstract Miguel explains the limitations and considerations of applying ML in robotics, contrasting its use against traditional control methods that offer exactness, which ML approaches generally approximate. He discusses the integration of computer vision and machine learning in sports for player movement tracking and performance analysis, highlighting collaborations with European football clubs and the role of artificial intelligence in strategic game analysis, akin to a coach's perspective. // Bio Miguel Fierro is a Principal Data Science Manager at Microsoft Spain, where he helps customers solve business problems using artificial intelligence. Previously, he was CEO and founder of Samsamia Technologies, a company that created a visual search engine for fashion items allowing users to find products using images instead of words, and founder of the Robotics Society of Universidad Carlos III, which developed different projects related to UAVs, mobile robots, humanoid robots, and 3D printers. Miguel has also worked as a robotics scientist at Universidad Carlos III of Madrid (UC3M) and King’s College London (KCL) and has collaborated with other universities like Imperial College London and IE University in Madrid. Miguel is an Electrical Engineer by UC3M, PhD in robotics by UC3M in collaboration with KCL, and graduated from MIT Sloan School of Management. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://miguelgfierro.com GitHub: https://github.com/miguelgfierro/RecSys at Spotify // Sanket Gupta // MLOps Podcast #232 - https://youtu.be/byH-ARJA4gkRecommenders joins LF AI & Data as new Sandbox project: https://cloudblogs.microsoft.com/opensource/2023/10/10/recommenders-joins-lf-ai-data-as-new-sandbox-project/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Miguel on LinkedIn: https://www.linkedin.com/in/miguelgfierro/ Timestamps: [00:00] Miguel's preferred coffee [00:11] Takeaways [02:25] Robotics [10:44] Simpler solutions over ML [15:11] Robotics and Computer Vision [19:15] Basketball object detection [22:43 - 23:50] Zilliz Ad [23:51] Mr. Recommenders and Recommender systems' common patterns [31:35] Embeddings and Feature Stores [42:34] Experiment ROI for leadership [47:17] Hi ROI investments [51:13] LLMs in Recommender Systems [54:51] Wrap up

Uber's Michelangelo: Strategic AI Overhaul and Impact // MLOps podcast #239 with Demetrios Brinkmann. Huge thank you to Weights & Biases for sponsoring this episode. WandB Free Courses - http://wandb.me/courses_mlops // Abstract Uber's Michelangelo platform has evolved significantly through three major phases, enhancing its capabilities from basic ML predictions to sophisticated uses in deep learning and generative AI. Initially, Michelangelo 1.0 faced several challenges such as a lack of deep learning support and inadequate project tiering. To address these issues, Michelangelo 2.0 and subsequently 3.0 introduced improvements like support for Pytorch, enhanced model training, and integration of new technologies like Nvidia’s Triton and Kubernetes. The platform now includes advanced features such as a Genai gateway, robust compliance guardrails, and a system for monitoring model performance to streamline and secure AI operations at Uber. // Bio At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios constantly learns and engages in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether analyzing the best paths forward, overcoming obstacles, or building Lego houses with his daughter. // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links From Predictive to Generative – How Michelangelo Accelerates Uber’s AI Journey blog post: https://www.uber.com/en-JP/blog/from-predictive-to-generative-ai/

Uber's Michelangelo: https://www.uber.com/en-JP/blog/michelangelo-machine-learning-platform/ The Future of Feature Stores and Platforms // Mike Del Balso & Josh Wills // MLOps Podcast # 186: https://youtu.be/p5F7v-w4EN0

Machine Learning Education at Uber // Melissa Barr & Michael Mui // MLOps Podcast #156: https://youtu.be/N6EbBUFVfO8 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Timestamps: [00:00] Uber's Michelangelo platform evolution analyzed in podcast [03:51 - 4:50] Weights & Biases Ad [05:57] Uber creates Michelangelo to streamline machine learning [07:44] Michelangelo platform's tech and flexible system [11:49] Uber Michelangelo platform adapted for deep learning [16:48] Uber invests in ML training for employees [19:08] Explanation of blog content, ML quality metrics [22:38] Michelangelo 2.0 prioritizes serving latency and Kubernetes [26:30] GenAI gateway manages model routing and costs [31:35] ML platform evolution, legacy systems, and maintenance [33:22] Team debates maintaining outdated tools or moving on [34:41] Please like, share, leave feedback, and subscribe to our MLOps channels! [34:57] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Matthew McClean is a Machine Learning Technology Leader with the leading Amazon Web Services (AWS) cloud platform. He leads the customer engineering teams at Annapurna ML helping customers adopt AWS Trainium and Inferentia for their Gen AI workloads. Kamran Khan, Sr Technical Business Development Manager for AWS Inferentina/Trianium at AWS. He has over a decade of experience helping customers deploy and optimize deep learning training and inference workloads using AWS Inferentia and AWS Trainium. AWS Tranium and Inferentia // MLOps podcast #238 with Kamran Khan, BD, Annapurna ML and Matthew McClean, Annapurna Labs Lead Solution Architecture at AWS. Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/ // Abstract Unlock unparalleled performance and cost savings with AWS Trainium and Inferentia! These powerful AI accelerators offer MLOps community members enhanced availability, compute elasticity, and energy efficiency. Seamlessly integrate with PyTorch, JAX, and Hugging Face, and enjoy robust support from industry leaders like W&B, Anyscale, and Outerbounds. Perfectly compatible with AWS services like Amazon SageMaker, getting started has never been easier. Elevate your AI game with AWS Trainium and Inferentia! // Bio Kamran Khan Helping developers and users achieve their AI performance and cost goals for almost 2 decades. Matthew McClean Leads the Annapurna Labs Solution Architecture and Prototyping teams helping customers train and deploy their Generative AI models with AWS Trainium and AWS Inferentia // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AWS Trainium: https://aws.amazon.com/machine-learning/trainium/ AWS Inferentia: https://aws.amazon.com/machine-learning/inferentia/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Kamran on LinkedIn: https://www.linkedin.com/in/kamranjk/ Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewmcclean/ Timestamps: [00:00] Matt's & Kamran's preferred coffee [00:53] Takeaways [01:57] Please like, share, leave a review, and subscribe to our MLOps channels! [02:22] AWS Trainium and Inferentia rundown [06:04] Inferentia vs GPUs: Comparison [11:20] Using Neuron for ML [15:54] Should Trainium and Inferentia go together? [18:15] ML Workflow Integration Overview [23:10] The Ec2 instance [24:55] Bedrock vs SageMaker [31:16] Shifting mindset toward open source in enterprise [35:50] Fine-tuning open-source models, reducing costs significantly [39:43] Model deployment cost can be reduced innovatively [43:49] Benefits of using Inferentia and Trainium [45:03] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/.

Benjamin Wilms is a developer and software architect at heart, with 20 years of experience. He fell in love with chaos engineering. Benjamin now spreads his enthusiasm and new knowledge as a speaker and author – especially in the field of chaos and resilience engineering. Retrieval Augmented Generation // MLOps podcast #237 with Benjamin Wilms, CEO & Co-Founder of Steadybit. Huge thank you to Amazon Web Services for sponsoring this episode. AWS - https://aws.amazon.com/ // Abstract How to build reliable systems under unpredictable conditions with Chaos Engineering. // Bio Benjamin has over 20 years of experience as a developer and software architect. He fell in love with chaos engineering 7 years ago and shares his knowledge as a speaker and author. In October 2019, he founded the startup Steadybit with two friends, focusing on developers and teams embracing chaos engineering. He relaxes by mountain biking when he's not knee-deep in complex and distributed code. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://steadybit.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Benjamin on LinkedIn: https://www.linkedin.com/in/benjamin-wilms/ Timestamps: [00:00] Benjamin's preferred coffee [00:28] Takeaways [02:10] Please like, share, leave a review, and subscribe to our MLOps channels! [02:53] Chaos Engineering tldr [06:13] Complex Systems for smaller Startups [07:21] Chaos Engineering benefits [10:39] Data Chaos Engineering trend [15:29] Chaos Engineering vs ML Resilience [17:57 - 17:58] AWS Trainium and AWS Infecentia Ad [19:00] Chaos engineering tests system vulnerabilities and solutions

[23:24] Data distribution issues across different time zones

[27:07] Expertise is essential in fixing systems

[31:01] Chaos engineering integrated into machine learning systems

[32:25] Pre-CI/CD steps and automating experiments for deployments

[36:53] Chaos engineering emphasizes tool over value

[38:58] Strong integration into observability tools for repeatable experiments

[45:30] Invaluable insights on chaos engineering

[46:42] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Tom Smoker is the cofounder of an early stage tech company empowering developers to create knowledge graphs within their RAG pipelines. Tom is a technical founder, and owns the research and development of knowledge graphs tooling for the company. Managing Small Knowledge Graphs for Multi-agent Systems // MLOps podcast #236 with Tom Smoker, Technical Founder of whyhow.ai. A big thank you to  @latticeflow  for sponsoring this episode! LatticeFlow - https://latticeflow.ai/ // Abstract RAG is one of the more popular use cases for generative models, but there can be issues with repeatability and accuracy. This is especially applicable when it comes to using many agents within a pipeline, as the uncertainty propagates. For some multi-agent use cases, knowledge graphs can be used to structurally ground the agents and selectively improve the system to make it reliable end to end. // Bio Technical Founder of WhyHow.ai. Did Masters and PhD in CS, specializing in knowledge graphs, embeddings, and NLP. Worked as a data scientist to senior machine learning engineer at large resource companies and startups. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models: https://arxiv.org/abs/2401.01313Understanding the type of Knowledge Graph you need — Fixed vs Dynamic Schema/Data: https://medium.com/enterprise-rag/understanding-the-type-of-knowledge-graph-you-need-fixed-vs-dynamic-schema-data-13f319b27d9e --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Tom on LinkedIn: https://www.linkedin.com/in/thomassmoker/ Timestamps: [00:00] Tom's preferred coffee [00:33] Takeaways [03:04] Please like, share, leave a review, and subscribe to our MLOps channels! [03:23] Academic Curiosity and Knowledge Graphs [05:07] Logician [05:53] Knowledge graphs incorporated into RAGs [07:53] Graphs & Vectors Integration [10:49] "Exactly wrong" [12:14] Data Integration for Robust Knowledge Graph [14:53] Structured and Dynamic Data [21:44] Scoped Knowledge Retrieval Strategies [28:01 - 29:32] LatticeFlow Ad [29:33] RAG Limitations and Solutions [36:10] Working on multi agents, questioning agent definition

[40:01] Concerns about performance of agent information transfer

[43:45] Anticipating agent-based systems with modular processes

[52:04] Balancing risk tolerance in company operations and control

[54:11] Using AI to generate high-quality, efficient content

[01:03:50] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

David Nunez, based in Santa Barbara, CA, US, is currently a Co-Founder and Partner at Abstract Group, bringing experience from previous roles at First Round Capital, Stripe, and Slab. Just when we Started to Solve Software Docs, AI Blew Everything Up // MLOps Podcast #235 with Dave Nunez, Partner of Abstract Group co-hosted by Jakub Czakon. Huge thank you to Zilliz for sponsoring this episode. Zilliz - https://zilliz.com/. // Abstract Over the previous decade, the recipe for making excellent software docs mostly converged on a set of core goals: Create high-quality, consistent content Use different content types depending on the task Make the docs easy to find For AI-focused software and products, the entire developer education playbook needs to be rewritten. // Bio Dave lives in Santa Barbara, CA with his wife and four kids. He started his tech career at various startups in Santa Barbara before moving to San Francisco to work at Salesforce. After Salesforce, he spent 2+ years at Uber and 5+ years at Stripe leading internal and external developer documentation efforts. In 2021, he co-authored Docs for Developers to help engineers become better writers. He's now a consultant, advisor, and angel investor for fast-growing startups. He typically invests in early-stage startups focusing on developer tools, productivity, and AI. He's a reading nerd, Lakers fan, and golf masochist. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.abstractgroup.co/ Book: docsfordevelopers.com About Dave: https://gamma.app/docs/Dave-Nunez-about-me-002doxb23qbblme?mode=doc https://review.firstround.com/investing-in-internal-documentation-a-brick-by-brick-guide-for-startups https://increment.com/documentation/why-investing-in-internal-docs-is-worth-it/

Writing to Learn paper by Peter Elbow: https://peterelbow.com/pdfs/Writing_for_Learning-Not_just_Demonstrating.PDF --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Dave on LinkedIn: https://www.linkedin.com/in/djnunez/ Connect with Kuba on LinkedIn: https://www.linkedin.com/in/jakub-czakon/?locale=en_US Timestamps: [00:00] Dave's preferred coffee [00:13] Introducing this episode's co-host, Kuba [00:36] Takeaways [02:55] Please like, share, leave a review, and subscribe to our MLOps channels! [03:23] Good docs, bad docs, and how to feel them [06:51] Inviting Dev docs and checks [10:36] Stripe's writing culture [12:42] Engineering team writing culture [14:15] Bottom-up tech writer change [18:31] Strip docs cult following [24:40] TriDocs Smart API Injection [26:42] User research for documentation [29:51] Design cues [32:15] Empathy-driven docs creation [34:28 - 35:35] Zilliz Ad [35:36] Foundational elements in documentation [38:23] Minimal infrastructure of information in "Read Me" [40:18] Measuring documentation with OKRs [43:58] Improve pages with Analytics [47:33] Google branded doc searches [48:35] Time to First Action [52:52] Dave's day in and day out and what excites him [56:01] Exciting internal documentation [59:55] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Cody Peterson has a diverse work experience in the field of product management and engineering. Cody is currently working as a Technical Product Manager at Voltron Data, starting from May 2023. Previously, they worked as a Product Manager at dbt Labs from July 2022 to March 2023. MLOps podcast #234 with Cody Peterson, Senior Technical Product Manager at Voltron Data | Ibis project // Open Standards Make MLOps Easier and Silos Harder. Huge thank you to Weights & Biases for sponsoring this episode. WandB Free Courses -http://wandb.me/courses_mlops // Abstract MLOps is fundamentally a discipline of people working together on a system with data and machine learning models. These systems are already built on open standards we may not notice -- Linux, git, scikit-learn, etc. -- but are increasingly hitting walls with respect to the size and velocity of data. Pandas, for instance, is the tool of choice for many Python data scientists -- but its scalability is a known issue. Many tools make the assumption of data that fits in memory, but most organizations have data that will never fit in a laptop. What approaches can we take? One emerging approach with the Ibis project (created by the creator of pandas, Wes McKinney) is to leverage existing "big" data systems to do the heavy lifting on a lightweight Python data frame interface. Alongside other open source standards like Apache Arrow, this can allow data systems to communicate with each other and users of these systems to learn a single data frame API that works across any of them. Open standards like Apache Arrow, Ibis, and more in the MLOps tech stack enable freedom for composable data systems, where components can be swapped out allowing engineers to use the right tool for the job to be done. It also helps avoid vendor lock-in and keep costs low. // Bio Cody is a Senior Technical Product Manager at Voltron Data, a next-generation data systems builder that recently launched an accelerator-native GPU query engine for petabyte-scale ETL called Theseus. While Theseus is proprietary, Voltron Data takes an open periphery approach -- it is built on and interfaces through open standards like Apache Arrow, Substrait, and Ibis. Cody focuses on the Ibis project, a portable Python dataframe library that aims to be the standard Python interface for any data system, including Theseus and over 20 other backends. Prior to Voltron Data, Cody was a product manager at dbt Labs focusing on the open source dbt Core and launching Python models (note: models is a confusing term here). Later, he led the Cloud Runtime team and drastically improved the efficiency of engineering execution and product outcomes. Cody started his carrer as a Product Manager at Microsoft working on Azure ML. He spent about 2 years on the dedicated MLOps product team, and 2 more years on various teams across the ML lifecycel including data, training, and inferencing. He is now passionate about using open source standards to break down the silos and challenges facing real world engineering teams, where engineering increasingly involves data and machine learning. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Ibis Project: https://ibis-project.org Apache Arrow and the “10 Things I Hate About pandas”: https://wesmckinney.com/blog/apache-arrow-pandas-internals/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Cody on LinkedIn: https://linkedin.com/in/codydkdc

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Syed Asad is an Innovator, Generative AI & Machine Learning Engineer, and a Champion for Ethical AI
MLOps podcast #233 with Syed Asad, Lead AI/ML Engineer at KiwiTech // Retrieval Augmented Generation. A big thank you to @ for sponsoring this episode! AWS - // Abstract Everything and anything around RAG. // Bio Currently Exploring New Horizons: Syed is diving deep into the exciting world of Semantic Vector Searches and Vector Databases. These innovative technologies are reshaping how we interact with and interpret vast data landscapes, opening new avenues for discovery and innovation. Specializing in Retrieval Augmented Generation (RAG): Syed's current focus also includes mastering Retrieval Augmented Generation Techniques (RAGs). This cutting-edge approach combines the power of information retrieval with generative models, setting new benchmarks in AI's capability and application. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://sanketgupta.substack.com/ Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584 Sanket's blogs on Medium in the past: https://medium.com/@sanket107 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Syed on LinkedIn: https://www.linkedin.com/in/syed-asad-76815246/ Timestamps: [00:00] Syed's preferred coffee [00:31] Takeaways [03:17] Please like, share, leave a review, and subscribe to our MLOps channels! [03:37] A production issue [07:37] CSV file handling risks [09:42] Embedding models not suitable [11:22] Inference layer experiments and use cases [14:00] AWS service handling the issue [17:35] Salad testing and insights [22:12] OpenAI vs Customization [24:30] Difference between Olama and VLLM [27:16] Fine-tuning of small LLMs [29:51] Evaluation framework [32:04] MLOps for efficient ML [37:12] Determining the pricing of tools [39:35] Manage Dependency Risk [40:27] Get in touch with Syed on LinkedIn [41:46] ML Engineers are now all AI Engineers [43:01] The hard framework [43:53] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Sanket works as a Senior Machine Learning Engineer at Spotify working on building end-to-end audio recommender systems. Models built by his team are used across Spotify in many different products including Discover Weekly and Autoplay. MLOps podcast #232 with Sanket Gupta, Senior Machine Learning Engineer at Spotify // RecSys at Spotify. A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/ // Abstract LLMs with foundational embeddings have changed the way we approach AI today. Instead of re-training models from scratch end-to-end, we instead rely on fine-tuning existing foundation models to perform transfer learning. Is there a similar approach we can take with recommender systems? In this episode, we can talk about: a) how Spotify builds and maintains large-scale recommender systems, b) how foundational user and item embeddings can enable transfer learning across multiple products, c) how we evaluate this system d) MLOps challenges with these systems // Bio Sanket works as a Senior Machine Learning Engineer on a team at Spotify building production-grade recommender systems. Models built by my team are being used in Autoplay, Daily Mix, Discover Weekly, etc. Currently, my passion is how to build systems to understand user taste - how do we balance long-term and short-term understanding of users to enable a great personalized experience. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://sanketgupta.substack.com/ Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584 Sanket's blogs on Medium in the past: https://medium.com/@sanket107 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sanket on LinkedIn: www.linkedin.com/in/sanketgupta107 Timestamps: [00:00] Sanket's preferred coffee [00:37] Takeaways [02:30] RecSys are RAGs [06:22] Evaluating RecSys parallel to RAGs [07:13] Music RecSys Optimization [09:46] Dealing with cold start problems [12:18] Quantity of models in the recommender systems [13:09] Radio models [16:24] Evaluation system [20:25] Infrastructure support [21:25] Transfer learning [23:53] Vector database features [25:31] Listening History Balance [26:35 - 28:06] LatticeFlow Ad [28:07] The beauty of embeddings [30:13] Shift to real-time recommendation [34:05] Vector Database Architecture Options [35:30] Embeddings drive personalized [40:16] Feature Stores vs Vector Databases [42:33] Spotify product integration strategy [45:38] Staying up to date with new features [47:53] Speed vs Relevance metrics [49:40] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Ryan Carson. CEO Founder for 20 years Built and sold 3 startups Helping build a global community of AI devs with Intel. MLOps podcast #231 with Ryan Carson, Senior AI Dev Community Lead at Intel Huge thank you to Zilliz for sponsoring this episode. Zilliz - https://zilliz.com/ // Abstract Ryan shares his professional journey, tracing his transition from building Treehouse to joining Intel. The conversation evolves into a deep dive into Carson's aspiration to democratize access to AI development. Furthermore, he expounds on the exciting prospects of new technology like Gaudi three, a new ASIC for AI workloads. Ryan emphasizes the need for driving competition in compute to lower prices and increase access, underlining the importance of associating individual work with company-based OKRs or KPIs. There is also a reflection on the essentiality of forging quality relationships in professional settings and aligning work with top-level OKRs. Discussion on the potential benefits of AI in constructing and maintaining professional interactions is explored. Touching upon practical applications of AI, they also delve into smaller projects, the possibility of one-person companies, and the role of AI for daily interactions. The episode concludes with an expression of optimism about technological advances shaping the future and an appreciation for the enlightening conversation. // Bio Ryan has been a founder, entrepreneur, and CEO for 20 years, successfully building, scaling, and selling three companies. He's passionate about empowering people to become developers and then connecting them together in a global community. After earning a degree in Computer Science in Colorado, Ryan moved to the UK and worked as a web developer. He then organized global tech conferences, hosting thousands of attendees and influential speakers such as Mark Zuckerberg, the founders of Android, Instagram, and Twitter, among others. His company also produced Twitter’s and Stack Overflow’s developer conferences. Following that, Ryan started an online Computer Science school. Under his leadership, the team grew to over 100 employees, educating more than 1,000,000 students. During this period, he secured $23 million in venture capital and earned recognition as Entrepreneur of the Year. Over the last two years Ryan dove deep into AI and LLMs. He built an educational proof-of-concept called maple.coach, which focuses on teaching Sales. The platform is built using technologies like Next.js, TypeScript, gpt-4, and Vercel. Outside of work, Ryan shares his life with his wife of 20 years and their two teenagers in Connecticut. They enjoy spending their free time sailing and taking walks with their Sheltie, Brinkley. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: ryancarson.com --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ryan on LinkedIn: https://www.linkedin.com/in/ryancarson/

Salman Avestimehr is a Dean's Professor, the inaugural director of the USC-Amazon Center for Secure and Trusted Machine Learning (Trusted AI), and director of the Information Theory and Machine Learning (vITAL) research lab. He is also the CEO and co-founder of FedML. MLOps podcast #230 with Salman Avestimehr, CEO & Founder of FedML, FedML Nexus AI: Your Generative AI Platform at Scale.

A big thank you to FEDML for sponsoring this episode! // Abstract FedML is your generative AI platform at scale to enable developers and enterprises to build and commercialize their own generative AI applications easily, scalably, and economically. Its flagship product, FedML Nexus AI, provides unique features in enterprise AI platforms, model deployment, model serving, AI agent APIs, launching training/Inference jobs on serverless/decentralized GPU cloud, experimental tracking for distributed training, federated learning, security, and privacy. // Bio Salman is a professor, the inaugural director of the USC-Amazon Center for Secure and Trusted Machine Learning (Trusted AI), and the director of the Information Theory and Machine Learning (vITAL) research lab at the Electrical and Computer Engineering Department and Computer Science Department of the University of Southern California. Salman is also the co-founder and CEO of FedML. He received his Ph.D. in Electrical Engineering and Computer Sciences from UC Berkeley in 2008. Salman does research in the areas of information theory, decentralized and federated machine learning, secure and privacy-preserving learning, and computing. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.avestimehr.com/ https://fedml.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Salman on LinkedIn: https://www.linkedin.com/company/fedml/ Timestamps: [00:00] AI Quality: First in-person conference on June 25 [01:28] Salman's preferred coffee [01:49] Takeaways [03:33] Please like, share, leave a review, and subscribe to our MLOps channels! [03:53] Challenges that inspired Salman's work [06:20] Controlled ownership [08:11] Dealing with data leakage and privacy problems [10:45] In-house ML Model Deployment [13:36] FEDML: Comprehensive Model Deployment [17:27] Integrating FEDML with Kubernetes [19:46] AI Evaluation Trends [24:37] Enhancing NLP with ML [25:48] FEDML: Canary, A/B, Confidence [29:36] FEDML customers [33:21] On-premise platform for secure data management

[37:16] Future prediction: data's crucial for better applications

[38:18] Maturity in evaluating and improving steps

[41:38] Focus on ownership

[45:12] Benefits of smaller models for specific use cases

[48:57] Verify sensitive tasks, trust quick, important mobile content creation

[51:50] Wrap up

Mohamed Elgendy is the Co-Founder & CEO at Kolena. Additionally, Mohamed Elgendy has had 1 past job as the Director Of Product and Engineering at Synapse Technology Corporation. Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ MLOps podcast #228 with Mohamed Elgendy, Co-founder & CEO of Kolena Inc., What is AI Quality? // Abstract Delve into the multifaceted concept of AI Quality. Demetrios and Mo explore the idea that AI quality is dependent on the specific domain, equitable to the difference in desired qualities between a $1 pen and a $100 pen. Mo underscores the performance of a product being in sync with its intended functionality and the absence of unknown risks as the pillars of AI Quality. They emphasize the need for comprehensive quality checks and adaptability of standards to differing product traits. Issues affecting edge deployments like latency are also highlighted. A deep dive into the formation of gold standards for AI, the nuanced necessities for various use cases, and the paramount need for collaboration among AI builders, regulators, and infrastructure firms form the core of the discussion. Elgendy brings to light their ambitious AI Quality Conference, aiming to set tangible, effective, but innovation-friendly Quality standards for AI. The dialogue also accentuates the urgent need for diversification and representation in the tech industry, the variability of standards and regulations, and the pivotal role of testing in AI and machine learning. The episode concludes with an articulate portrayal of how enhanced testing can streamline the entire process of machine learning. // Bio Mohamed is the Co-founder & CEO of Kolena and the author of the book “Deep Learning for Vision Systems”. Previously, he built and managed AI/ML organizations at Amazon, Twilio, Rakuten, and Synapse. Mohamed regularly speaks at AI conferences like Amazon's DevCon, O'Reilly's AI conference, and Google's I/O. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: www.kolena.io Deep Learning for Vision Systems book: https://www.amazon.com/Learning-Vision-Systems-Mohamed-Elgendy/dp/1617296198/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mo on LinkedIn: https://www.linkedin.com/in/moelgendy/ Timestamps: [00:00] Mo's preferred coffee [00:07] Takeaways [02:52] See you all in San Francisco on June 25! [03:04] Please like, share, leave a review, and subscribe to our MLOps channels! [03:22] AI Quality in Mo's eyes [08:36] Quality Standards for Software [14:11] Common Chatbot Functionality [19:20] The Birth of Innovation [24:27] Transforming Insights into Standards [30:27] Testing: One step to quality [34:58] Two different data points to be harmonized [37:29] Model cards [39:12] Test Coverage Democratizes Collaboration [42:55] Representation matters [44:50] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com

Simon Karasik⁠ is a proactive and curious ML Engineer with 5 years of experience. Developed & deployed ML models at WEB and Big scale for Ads and Tax. Huge thank you to Nebius AI for sponsoring this episode. Nebius AI - https://nebius.ai/ MLOps podcast #228 with Simon Karasik, Machine Learning Engineer at Nebius AI, Handling Multi-Terabyte LLM Checkpoints. // Abstract The talk provides a gentle introduction to the topic of LLM checkpointing: why is it hard, how big are the checkpoints. It covers various tips and tricks for saving and loading multi-terabyte checkpoints, as well as the selection of cloud storage options for checkpointing. // Bio Full-stack Machine Learning Engineer, currently working on infrastructure for LLM training, with previous experience in ML for Ads, Speech, and Tax. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Simon on LinkedIn: https://www.linkedin.com/in/simon-karasik/ Timestamps: [00:00] Simon preferred beverage [01:23] Takeaways [04:22] Simon's tech background [08:42] Zombie models garbage collection [10:52] The road to LLMs [15:09] Trained models Simon worked on [16:26] LLM Checkpoints [20:36] Confidence in AI Training [22:07] Different Checkpoints [25:06] Checkpoint parts [29:05] Slurm vs Kubernetes [30:43] Storage choices lessons [36:02] Paramount components for setup [37:13] Argo workflows [39:49] Kubernetes node troubleshooting [42:35] Cloud virtual machines have pre-installed mentoring [45:41] Fine-tuning [48:16] Storage, networking, and complexity in network design [50:56] Start simple before advanced; consider model needs. [53:58] Join us at our first in-person conference on June 25 all about AI Quality

Sol Rashidi is an esteemed executive, leader, and influencer within the AI, Data, and Technology space.  Having helped IBM launch Watson in 2011 as one of the earliest world applications of Artificial Intelligence, Sol has pioneered some of the early advancements of space.  Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ Huge thank you to  @WeightsBiases  for sponsoring this episode. WandB Free Courses - http://wandb.me/courses_mlops MLOps podcast #227 with Sol Rashidi, CEO & Co-Founder of ExecutiveAI, Leading Enterprise Data Teams. // Abstract In the dynamic landscape of MLOps and data leadership, Sol shares invaluable insights on building successful teams and driving impactful projects. In this podcast episode, Sol delves into the importance of prioritizing relationships, introduces a pragmatic "Wrong Use Cases Formula" to streamline project prioritization, and emphasizes the critical role of effective communication in data leadership. Her wealth of experience and practical advice provide a roadmap for navigating the complexities of MLOps and leading data-driven initiatives to success. // Bio With eight (8) patents granted, 21 filed, and received awards that include: "Top 100 AI People" 2023 "The Top 75 Innovators of 2023" "Top 65 Most Influential Women in 2023" "Forbes AI Maverick of the 21st Century" 2022 “Top 10 Global Women in AI & Data”, 2023 "Top AI 100 Award", 2023 “50 Most Powerful Women in Tech”, 2022 “Global 100 Power List” - 2021, 2022, 2023 “Top 20 CDOs Globally” - 2022 "Chief Analytics Officer of the Year" - 2022 "Isomer Innovators of the Year" - 2021, 2022, 2023 "Top 100 Innovators in Data & Analytics” - 2020, 2021, 2022, 2023 "Top 100 Women in Business" - 2022 Sol is an energetic business executive and a goal-oriented technologist, skilled at coupling her technical acumen with story-telling abilities to articulate business value with both startups and Fortune 100's who are leaning into data, AI, and technology as a competitive advantage while wanting to preserve the legacy in which they were founded upon. Sol has served as a C-Suite member across several Fortune 100 & Fortune 500 companies including: Chief Analytics Officer - Estee Lauder Chief Data & Analytics Officer - Merck Pharmaceuticals EVP, Chief Data Officer - Sony Music Chief Data & AI Officer - Royal Caribbean Cruise Lines Sr. Partner leading the Digital & Innovation Practice- Ernsty & Young Partner leading Watson Go-To-Market & Commercialization - IBM Sol now serves as the CEO of ExecutiveAI LLC. A company dedicated to democratizing Artificial Intelligence for Humanity and is considered an outstanding and influential business leader who is influencing the space traveling the world as a keynote speaker, and serving as the bridge between established Gen1.0 markets and those evolving into 4.0. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Sol's Book will be out on April 30, 2024 Your AI Survival Guide: Scraped Knees, Bruised Elbows, and Lessons Learned from Real-World AI Deployments: https://www.amazon.com/Your-Survival-Guide-Real-World-Deployments/dp/1394272634?ref_=ast_author_mpb --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sol on LinkedIn: https://www.linkedin.com/in/sol-rashidi-a672291/

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Chad Sanderson is passionate about data quality, and fixing the muddy relationship between data producers and consumers. He is a former Head of Data at Convoy, a LinkedIn writer, and a published author. He lives in Seattle, Washington, and is the Chief Operator of the Data Quality Camp. Huge thank you to @amazonwebservices for sponsoring this episode. AWS - https://aws.amazon.com/ MLOps podcast #226 with Chad Sanderson, CEO & Co-Founder of Gable, The Rise of Modern Data Management. // Abstract In this session, Chad Sanderson, CEO of Gable.ai and author of the upcoming O’Reilly book: "Data Contracts," tackles the necessity of modern data management in an age of hyper iteration, experimentation, and AI. He will explore why traditional data management practices fail and how the cloud has fundamentally changed data development. The talk will cover a modern application of data management best practices, including data change detection, data contracts, observability, and CI/CD tests, and outline the roles of data producers and consumers. Attendees will leave with a clear understanding of modern data management's components and how to leverage them for better data handling and decision-making. // Bio Chad Sanderson, CEO of Gable.ai, is a prominent figure in the data tech industry, having held key data positions at leading companies such as Convoy, Microsoft, Sephora, Subway, and Oracle. He is also the author of the upcoming O'Reilly book, "Data Contracts” and writes about the future of data infrastructure, modeling, and contracts in his newsletter “Data Products.” // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AWS Trainium and Inferentia: https://aws.amazon.com/machine-learning/trainium/ https://aws.amazon.com/machine-learning/inferentia/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Chad on LinkedIn: https://www.linkedin.com/in/chad-sanderson/

Patrick Beukema has a Ph.D. in neuroscience and has worked on AI models for brain decoding, which analyzes the brain's activity to decipher what people are seeing and thinking. Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ Huge thank you to LatticeFlow for sponsoring this episode. LatticeFlow - https://latticeflow.ai/ MLOps podcast #225 with Patrick Beukema, Head / Technical Lead of the Environmental AI, Applied Science Organization at AI2, Beyond AGI, Can AI Help Save the Planet? // Abstract AI will play a central role in solving some of our greatest environmental challenges. The technology that we need to solve these problems is in a nascent stage -- we are just getting started. For example, the combination of remote sensing (satellites) and high-performance AI operating at a global scale in real-time unlocks unprecedented avenues to new intelligence. MLOPs is often overlooked on AI teams, and typically there is a lot of friction in integrating software engineering best practices into the ML/AI workflow. However, performance ML/AI depends on extremely tight feedback loops from the user back to the model that enables high iteration velocity and ultimately continual improvement. We are making progress but environmental causes need your help. Join us fight for sustainability and conservation. // Bio Patrick is a machine learning engineer and scientist with a deep passion for leveraging artificial intelligence for social good. He currently leads the environmental AI team at the Allen Institute for Artificial Intelligence (AI2). His professional interests extend to enhancing scientific rigor in academia, where he is a strong advocate for the integration of professional software engineering practices to ensure reliability and reproducibility in academic research. Patrick holds a Ph.D. from the Center for Neuroscience at the University of Pittsburgh and the Center for the Neural Basis of Cognition at Carnegie Mellon University, where his research focused on neural plasticity and accelerated learning. He applied this expertise to develop state-of-the-art deep learning models for brain decoding of patient populations at a startup, later acquired by BlackRock. His earlier academic work spanned research on recurrent neural networks, causal inference, and ecology and biodiversity. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Variety of relevant papers/talks/links on Patrick's website: https://pbeukema.github.io/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Patrick on LinkedIn: https://www.linkedin.com/in/plbeukema/ Timestamps: [00:00] AI Quality Conference [01:29] Patrick's preferred coffee [02:00] Takeaways [04:14] Learning how to learn journey [07:04] Patrick's day to day [08:39] Environmental AI [11:07] Environmental AI models [14:35] Nature Inspires Scientific Advances [18:11] R&D [24:58] Iterative Feedback-Driven Development [26:37 - 28:07] LatticeFlow Ad [33:58] Balancing Metrics for Success [38:16] Model Retraining Pipeline [44:11] Series Models: Versatility [45:57] Edge Models Enhance Output [50:22] Custom Models for Specific Data [53:53] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Verena Weber believes that GenAI is going to transform the way we work and interact with devices. Her mission is to help companies prepare for this transformation. She has strong expertise in NLP and over 7 years of experience in Machine Learning. Huge thank you to  @zilliz  for sponsoring this episode. Zilliz - https://zilliz.com/ MLOps podcast #224 with Verena Weber, Generative AI Consultant at Verena Weber, GenAI in Production - Challenges and Trends. // Abstract The goal of this talk is to provide insights into challenges for Generative AI in production as well as trends aiming to solve some of these challenges. The challenges and trends Verena see are: Model size and moving towards mixture of experts architectures context window - new breakthroughs for context lengths from unimodality to multimodality, next step large action models? regulation in form of the EU AI Act Verena uses the differences between Gemini 1.0 and Gemini 1.5 to exemplify some of these trends. // Bio Verena leverages GenAI in natural language to elevate business competitiveness and navigate its transformative impact. Her varied experience in multiple roles and sectors underpins her ability to extract business value from AI, blending deep technical expertise with strong business acumen. Post-graduation, she consulted in Data Science at Deloitte and then advanced her skills in NLP, Deep Learning, and GenAI as a Research Scientist at Alexa team, Amazon. Passionate about gender diversity in tech, she mentors women to thrive in this field. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: verenaweber.de Sign up for Verena's newsletter: https://verenas-newsletter-63558b.beehiiv.com/ Zilliz - https://zilliz.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Verena on LinkedIn: https://www.linkedin.com/in/verena-weber-134178b9/ Timestamps: [00:00] AI Quality Conference [01:33] Verena's preferred coffee [02:15] Takeaways [06:33] Ski Person of Influence [11:31] Verena's background in the last 5-10 years [14:24] Tech Evolution: Rapid Transformation [18:13] Working at Amazon and key challenges [20:10] Research-inspired suggestions [22:21] AI Updates Impact Workflows [22:52] Alexa Query Distribution Analysis [24:06] Innovative Solutions for Alexa [25:27] Robust T5 Data Prompting [27:38] Audio Data Quality Challenges [28:21-29:28] Zilliz ad [29:28] Alexa data transcription and data cleaning

[35:38] Considering needs, costs, and complexity

[37:44] ChatGPt is not ideal for classification

[39:32] Comparison of model building using TF, IDF

[45:08] Struggle to boost diversity in conference speakers

[47:30] Creating safe environments helps underrepresented individuals participate

[48:29] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

MLOps Coffee Sessions Special episode with Databricks, Introducing DBRX: The Future of Language Models, fueled by our Premium Brand Partner, Databricks. DBRX is designed to be especially capable of a wide range of tasks and outperforms other open LLMs on standard benchmarks. It also promises to excel at code and math problems, areas where others have struggled. Our panel of experts will get into the technical nuances, potential applications, and implications of DBRx for businesses, developers, and the broader tech community. This session is a great opportunity to hear from insiders about how DBRX's capabilities can benefit you. // Bio Denny Lee - Co-host Denny Lee is a long-time Apache Spark™ and MLflow contributor, Delta Lake maintainer, and a Sr. Staff Developer Advocate at Databricks. A hands-on distributed systems and data sciences engineer with extensive experience developing internet-scale data platforms and predictive analytics systems. He has previously built enterprise DW/BI and big data systems at Microsoft, including Azure Cosmos DB, Project Isotope (HDInsight), and SQL Server. Davis Blalock Davis Blalock is a research scientist and the first employee at MosaicML. He previously worked at PocketSonics (acquired 2013) and completed his PhD at MIT, where he was advised by John Guttag. He received his M.S. from MIT and his B.S. from the University of Virginia. He is a Qualcomm Innovation Fellow, NSF Graduate Research Fellow, and Barry M. Goldwater Scholar. He is also the author of Davis Summarizes Papers, one of the most widely-read machine learning newsletters. Bandish Shah Bandish Shah is an Engineering Manager at MosaicML/Databricks, where he focuses on making generative AI training and inference efficient, fast, and accessible by bridging the gap between deep learning, large-scale distributed systems, and performance computing. Bandish has over a decade of experience building systems for machine learning and enterprise applications. Prior to MosaicML, Bandish held engineering and development roles at SambaNova Systems where he helped develop and ship the first RDU systems from the ground up, and Oracle where he worked as an ASIC engineer for SPARC-based enterprise servers. Abhi Venigalla Abhi is an NLP architect working on helping organizations build their own LLMs using Databricks. Joined as part of the MosaicML team and used to work as a researcher at Cerebras Systems. Ajay Saini Ajay is an engineering manager at Databricks leading the GenAI training platform team. He was one of the early engineers at MosaicML (acquired by Databricks) where he first helped build and launch Composer (an open source deep learning training framework) and afterwards led the development of the MosaicML training platform which enabled customers to train models (such as LLMs) from scratch on their own datasets at scale. Prior to MosaicML, Ajay was co-founder and CEO of Overfit, an online personal training startup (YC S20). Before that, Ajay worked on ML solutions for ransomware detection and data governance at Rubrik. Ajay has both a B.S. and MEng in computer science with a concentration in AI from MIT. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.databricks.com/ Databricks DBRX: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ // Abstract Dive into the challenges of scaling AI models from Minimum Viable Product (MVP) to full production. The panel emphasizes the importance of continually updating knowledge and data, citing examples like teaching AI systems nuanced concepts and handling brand name translations.

User feedback's role in model training, alongside evaluation steps like human annotation and heuristic-based assessment, was highlighted.

The speakers stressed the necessity of tooling for user evaluation, version control, and regular performance updates. Insights on in-house and external tools for annotation and evaluation were shared, providing a comprehensive view of the complexities involved in scaling AI models. // Bio Alex Volkov - Moderator Alex Volkov is an AI Evangelist at Weights & Biases, celebrated for his expertise in clarifying the complexities of AI and advocating for its beneficial uses. He is the founder and host of ThursdAI, a weekly newsletter, and podcast that explores the latest in AI, its practical applications, open-source, and innovation. With a solid foundation as an AI startup founder and 20 years in full-stack software engineering, Alex offers a deep well of experience and insight into AI innovation. Eric Peter Product management leader and 2x founder with experience in enterprise products, data, and machine learning. Currently building tools for generative AI @Databricks. Donné Stevenson Focused on building AI-powered products that give companies the tools and expertise needed to harness to power of AI in their respective fields. Phillip Carter Phillip is on the product team at Honeycomb where he works on a bunch of different developer tooling things. He's an OpenTelemetry maintainer -- chances are if you've read the docs to learn how to use OTel, you've read his words. He's also Honeycomb's (accidental) prompt engineering expert by virtue of building and shipping products that use LLMs. In a past life, he worked on developer tools at Microsoft, helping bring the first cross-platform version of .NET into the world and grow to 5 million active developers. When not doing computer stuff, you'll find Phillip in the mountains riding a snowboard or backpacking in the Cascades. Andrew Hoh Andrew Hoh is the President and Co-Founder of LastMile AI. Previously, he was a Group PM Manager at Meta AI, driving product for their AI Platform. Previously, he was the Product Manager for the Machine Learning Infrastructure team at Airbnb and a founding team member of Azure Cosmos DB, Microsoft Azure's distributed NoSQL database. He graduated with a BA in Computer Science from Dartmouth College. A big thank you to our Premium Sponsors,  @Databricks  and  @baseten  for their generous support! // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

Shane Morris is now a Senior Executive Advisor at Devis. Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ Huge thank you to  @WeightsBiases  for sponsoring this episode. WandB Free Courses - https://wandb.ai/telidavies/ml-news/reports/Introducing-W-B-MLOps-Courses-Free-Course-Effective-MLOps-Model-Development--VmlldzozMDk2ODA2 MLOps podcast #223 with Shane Morris, Senior Executive Advisor of Devis, Data Engineering in the Federal Sector. // Abstract Let's focus on autonomous systems rather than automation, and then super-narrow it down to smaller, cheaper, and more accessible autonomous systems. // Bio Former music and entertainment data and software person somehow moves into defense and national security, with hilarious and predictable results. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI Quality in Person Conference: https://www.aiqualityconference.com/ Website: https://shanemorris.sucks TikTok: https://www.tiktok.com/@shanemorrisdotsucks WandB Free Courses - https://wandb.ai/telidavies/ml-news/reports/Introducing-W-B-MLOps-Courses-Free-Course-Effective-MLOps-Model-Development--VmlldzozMDk2ODA2 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Shane on LinkedIn: https://www.linkedin.com/in/shanetollmanmorris/

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

⁠Peter Guagenti⁠ is an accomplished business builder and entrepreneur with expertise in strategy, product development, marketing, sales, and operations. Peter has helped build multiple successful start-ups to exits, fueling high growth in each company along the way. He brings a broad perspective, deep problem-solving skills, the ability to drive innovation amongst teams, and a proven ability to convert strategy into action -- all backed up by a history of delivering results. Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/ MLOps podcast #222 with Peter Guagenti, President & CMO of Tabnine - What Business Stakeholders Want to See from the ML Teams. // Abstract Peter Guagenti shares his expertise in the tech industry, discussing topics from managing large-scale tech legacy applications and data experimentation to the evolution of the Internet. He returns to his history of building and transforming businesses, such as his work in the early 90s for People magazine's website and his current involvement in AI development for software companies. Guagenti discusses the use of predictive modeling in customer management and emphasizes the importance of re-architecting solutions to fit customer needs. He also delves deeper into the AI tools' effectiveness in software development and the value of maintaining privacy. Guagenti sees a bright future in AI democratization and shares his company's development of AI coding assistants. Discussing successful entrepreneurship, Guagenti highlights balancing technology and go-to-market strategies and the value of failing fast. // Bio Peter Guagenti is the President and Chief Marketing Officer at Tabnine. Guagenti is an accomplished business leader and entrepreneur with expertise in strategy, product development, marketing, sales, and operations. He most recently served as chief marketing officer at Cockroach Labs, and he previously held leadership positions at SingleStore, NGINX (acquired by F5 Networks), and Acquia (acquired by Vista Equity Partners). Guagenti also serves as an advisor to a number of visionary AI and data companies including DragonflyDB, Memgraph, and Treeverse. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI Quality in Person Conference: https://www.aiqualityconference.com/ Measuring the impact of GitHub Copilot Survey: https://resources.github.com/learn/pathways/copilot/essentials/measuring-the-impact-of-github-copilot/ AWS Trainium and Inferentia: https://aws.amazon.com/machine-learning/trainium/ https://aws.amazon.com/machine-learning/inferentia/AI coding assistants: 8 features enterprises should seek: https://www.infoworld.com/article/3694900/ai-coding-assistants-8-features-enterprises-should-seek.htmlCareers at Tabnine: https://www.tabnine.com/careers --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Peter on LinkedIn: https://www.linkedin.com/in/peterguagenti/

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Amritha Arun Babu Mysore has been an expert in the field of consumer electronics, software products, and online marketplaces for the past 15 years. She has experience developing supply chains from the ground up, delivering AI-based products to millions of users, and advocating for ethical AI across Amazon, Wayfair, Salesforce, and NetApp.

Abhik Choudhury is a Senior Analytics Managing Consultant and Data Scientist with 11 years of experience in designing and implementing scalable data solutions for organizations across various industries. Huge thank you to  @latticeflow  for sponsoring this episode. LatticeFlow - https://latticeflow.ai/ MLOps podcast #221 with Amritha Arun Babu Mysore, ML Product Leader at Klaviyo and Abhik Choudhury, Managing Consultant Analytics at IBM, MLOps - Design Thinking to Build ML Infra for ML and LLM Use Cases. // Abstract As machine learning (ML) and large language models (LLMs) continue permeating industries, robust ML infrastructure and operations (ML Ops) are crucial to deploying these AI systems successfully. This podcast discusses best practices for building reusable, scalable, and governable ML Ops architectures tailored to ML and LLM use cases. // Bio Amritha Arun Babu Mysore Amritha is an accomplished technology leader with over 12 years of experience spearheading product innovation and strategic initiatives at both large enterprises and rapid-growth startups. Leveraging her background in engineering, supply chain, and business, Amritha has led high-performing teams to deliver transformative solutions solving complex challenges. She has driven product road mapping, requirements analysis, system design, and launch execution for advanced platforms in domains like machine learning, logistics, and e-commerce. Abhik Choudhury Abhik is a Senior Analytics Managing Consultant and Data Scientist with 11 years of experience in designing and implementing scalable data solutions for organizations across various industries. Throughout his career, Abhik developed a strong understanding of AI/ML, Cloud computing, database management systems, data modeling, ETL processes, and Big Data Technologies. Abhik's expertise lies in leading cross-functional teams and collaborating with stakeholders at all levels to drive data-driven decision-making in longitudinal pharmacy and medical claims and wholesale drug distribution areas. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI Quality in Person Conference in collaboration with Kolena: https://www.aiqualityconference.com/ LatticeFlow website: https://latticeflow.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abhik on LinkedIn: https://www.linkedin.com/in/abhik-choudhury-35450058 Connect with Amritha on LinkedIn: https://www.linkedin.com/in/amritha-arun-babu-a2273729/

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Demetrios Brinkmann is the founder of the MLOps Community. Brinkmann fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and Machine Learning. Huge thank you to Weights & Biases for sponsoring this episode. Weights & Biases - https://wandb.ai/site MLOps podcast #220 with our very own Founder of MLOps Community, Demetrios Brinkmann, Looking Back on 4 Years of the MLOps Community. // Abstract In this lively podcast episode, Mihail Eric hosts Demetrios Brinkmann, the founder of the MLOps Community, discussing its origin, structure, and challenges. Demetrios shares amusing tales of job hunting on LinkedIn and building the community despite lacking technical expertise, emphasizing the value of sharing and humor. They delve into the practicalities of hosting events, transitioning from self-funded to sponsorship-based, and tease upcoming activities with renowned speakers. Mihail and Demetrios explore job dynamics, the importance of sustained relationships, and diverse engagement methods like newsletters and volunteering. Demetrios reflects on his journey to Germany post-company closure, envisioning a global hub for AI learning, embodying the community's mission. // Bio At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building Lego houses with his daughter. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI Quality in Person Conference in collaboration with Kolena: https://www.aiqualityconference.com/ Weights & Biases Free Course: https://wandb.ai/telidavies/ml-news/reports/Introducing-W-B-MLOps-Courses-Free-Course-Effective-MLOps-Model-Development--VmlldzozMDk2ODA2What Does Best in Class AI/ML Governance Look Like in Fin Services? // Charles Radclyffe // MLOps #2: https://youtu.be/l52sRMVPVk0 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/ Timestamps: [00:00] Demetrios preferred coffee and bizarre listening [01:44] The MLOps Community Brainchild [04:22] The MLOps Community today [07:15] AI Quality in Person Conference on June 25th! [08:42] Community Quality [10:00] Community Learnings and the Genesis [17:55] The 600 Mark [20:15] The Feedback form [22:52] Demetrios' Journey and Learnings [29:01] Building full tolerance [29:55] Weights & Biases Free Course Ad [34:52] Building community involvement for professional success and networking [38:52] Balance in Community Growth [43:56] Collection of volunteers [49:00] Events Challenges [53:28] The future of MLOps Community [59:40] "Caveman" lifestyle choice [1:00:45] Stronger Hallucinogen [1:02:30] Wrap up

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Huge thank you to ⁠Databricks⁠ AI for sponsoring this episode. Databricks - http://databricks.com/

Bandish Shah is an Engineering Manager at MosaicML/Databricks, where he focuses on making generative AI training and inference efficient, fast, and accessible by bridging the gap between deep learning, large-scale distributed systems, and performance computing.

Davis Blalock is a Research Scientist and the first employee of Mosaic ML: a GenAI startup acquired for $1.3 billion by Databricks. MLOps podcast #219 with Databricks' Engineering Manager, Bandish Shah and Research Scientist Davis Blalock, The Art and Science of Training Large Language Models. // Abstract What's hard about language models at scale? Turns out...everything. MosaicML's Davis and Bandish share war stories and lessons learned from pushing the limits of LLM training and helping dozens of customers get LLMs into production. They cover what can go wrong at every level of the stack, how to make sure you're building the right solution, and some contrarian takes on the future of efficient models. // Bio Bandish Shah Bandish Shah is an Engineering Manager at MosaicML/Databricks, where he focuses on making generative AI training and inference efficient, fast, and accessible by bridging the gap between deep learning, large-scale distributed systems, and performance computing. Bandish has over a decade of experience building systems for machine learning and enterprise applications. Prior to MosaicML, Bandish held engineering and development roles at SambaNova Systems where he helped develop and ship the first RDU systems from the ground up, and Oracle where he worked as an ASIC engineer for SPARC-based enterprise servers. Davis Blalock Davis Blalock is a research scientist at MosaicML. He completed his PhD at MIT, advised by Professor John Guttag. His primary work is designing high-performance machine learning algorithms. He received his M.S. from MIT and his B.S. from the University of Virginia. He is a Qualcomm Innovation Fellow, NSF Graduate Research Fellow, and Barry M. Goldwater Scholar. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links

AI Quality In-person Conference: AI Quality in Person Conference: https://www.aiqualityconference.com/ Website: http://databricks.com/ Davis Summarizes Papers ⁠Newsletter signup link Davis' Newsletters: Learning to recognize spoken words from five unlabeled examples in under two seconds: https://arxiv.org/abs/1609.09196 Training on data at 5GB/s in a single thread: https://arxiv.org/abs/1808.02515 Nearest-neighbor searching through billions of images per second in one thread with no indexing: https://arxiv.org/abs/1706.10283 Multiplying matrices 10-100x faster than a matrix multiply (with some approximation error): https://arxiv.org/abs/2106.10860 Hidden Technical Debt in Machine Learning Systems: https://proceedings.neurips.cc/paper_files/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Davis on LinkedIn: https://www.linkedin.com/in/dblalock/ Connect with Bandish on LinkedIn: https://www.linkedin.com/in/bandish-shah/

// Abstract Diego, David, Ads, and Katharine, bring to light the risks, vulnerabilities, and evolving security landscape of machine learning as we venture into the AI-driven future. They underscore the importance of education in managing AI risks and the critical role privacy engineering plays in this narrative. They explore the legal and ethical implications of AI technologies, fostering a vital conversation on the balance between utility and privacy. // Bio Diego Oppenheimer - Moderator Diego Oppenheimer is a serial entrepreneur, product developer and investor with an extensive background in all things data. Currently, he is a Partner at Factory a venture fund specialized in AI investments as well as a co-founder at Guardrails AI. Previously he was an executive vice president at DataRobot, Founder and CEO at Algorithmia (acquired by DataRobot) and shipped some of Microsoft’s most used data analysis products including Excel, PowerBI and SQL Server. Diego is active in AI/ML communities as a founding member and strategic advisor for the AI Infrastructure Alliance and MLops.Community and works with leaders to define AI industry standards and best practices. Diego holds a Bachelor's degree in Information Systems and a Masters degree in Business Intelligence and Data Analytics from Carnegie Mellon University. Ads Dawson A mainly self-taught, driven, and motivated proficient application, network infrastructure & cyber security professional holding over eleven years experience from start-up to large-size enterprises leading the incident response process and specializing in extensive LLM/AI Security, Web Application Security and DevSecOps protecting REST API endpoints, large-scale microservice architectures in hybrid cloud environments, application source code as well as EDR, threat hunting, reverse engineering, and forensics. Ads have a passion for all things blue and red teams, be that offensive & API security, automation of detection & remediation (SOAR), or deep packet inspection for example. Ads is also a networking veteran and love a good PCAP to delve into. One of my favorite things at Defcon is hunting for PWNs at the "Wall of Sheep" village and inspecting malicious payloads and binaries. Katharine Jarmul Katharine Jarmul is a privacy activist and data scientist whose work and research focuses on privacy and security in data science workflows. She recently authored Practical Data Privacy for O'Reilly and works as a Principal Data Scientist at Thoughtworks. Katharine has held numerous leadership and independent contributor roles at large companies and startups in the US and Germany -- implementing data processing and machine learning systems with privacy and security built in and developing forward-looking, privacy-first data strategy. David Haber David has started and grown several technology companies. He developed safety-critical AI in the healthcare space and for autonomous flight. David has educated thousands of people and Fortune 500 companies on the topic of AI. Outside of work, he loves to spend time with his family and enjoys training for the next Ironman. A big thank you to our Premium Sponsors,  @Databricks  and  @baseten  for their generous support! // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

Frank Liu is the Director of Operations & ML Architect at Zilliz, where he serves as a maintainer for the Towhee open-source project. Jiang Chen is the Head of AI Platform and Ecosystem at Zilliz. Yujian Tang is a developer advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. MLOps Coffee Sessions Special episode with Zilliz, Why Purpose-built Vector Databases Matter for Your Use Case, fueled by our Premium Brand Partner, Zilliz. Engineering deep-dive into the world of purpose-built databases optimized for vector data. In this live session, we explore why non-purpose-built databases fall short in handling vector data effectively and discuss real-world use cases demonstrating the transformative potential of purpose-built solutions. Whether you're a developer, data scientist, or database enthusiast, this virtual roundtable offers valuable insights into harnessing the full potential of vector data for your projects. // Bio Jiang Chen Frank Liu is Head of AI & ML at Zilliz, with over eight years of industry experience in machine learning and hardware engineering. Before joining Zilliz, Frank co-founded Orion Innovations, an IoT startup based in Shanghai, and worked as an ML Software Engineer at Yahoo in San Francisco. He presents at major industry events like the Open Source Summit and writes tech content for leading publications such as Towards Data Science and DZone. His passion for ML extends beyond the workplace; in his free time, he trains ML models and experiments with unique architectures. Frank holds MS and BS degrees in Electrical Engineering from Stanford University. Frank Liu Jiang Chen is the Head of AI Platform and Ecosystem at Zilliz. With years of experience in data infrastructures and information retrieval, Jiang previously served as a tech lead and product manager for Search Indexing at Google. Jiang holds a Master's degree in Computer Science from the University of Michigan, Ann Arbor. Yujian Tang Yujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published to conferences including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://zilliz.com/ Neural Priming for Sample-Efficient Adaptation: https://arxiv.org/abs/2306.10191LIMA: Less Is More for Alignment: https://arxiv.org/abs/2305.11206ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT: https://arxiv.org/abs/2004.12832 Milvus Vector Database by Zilliz: https://zilliz.com/what-is-milvus --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Timestamps: [00:00] Demetrios' musical intro [04:36] Vector Databases vs. LLMs [07:51] Relevance Over Speed [12:55] Pipelines [16:19] Vector Databases Integration Benefits [26:42] Database Diversity Market [27:38] Milus vs. Pinecone [30:22] Vector DB for Training & Deployment [34:32] Future proof of AI applications [45:16] Data Size and Quality [48:53] ColBERT Model [54:25] Vector Data Consistency Best Practices [57:24] Wrap up

Huge thank you to LatticeFlow AI for sponsoring this episode. LatticeFlow AI - https://latticeflow.ai/.Dr. Petar Tsankov is a researcher and entrepreneur in the field of Computer Science and Artificial Intelligence. MLOps podcast #218 with Petar Tsankov, Co-Founder and CEO at LatticeFlow AI, A Decade of AI Safety and Trust. // Abstract // Bio Co-founder & CEO at LatticeFlow AI, building the world's first product enabling organizations to build performant, safe, and trustworthy AI systems. Before starting LatticeFlow AI, Petar was a senior researcher at ETH Zurich working on the security and reliability of modern systems, including deep learning models, smart contracts, and programmable networks. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://latticeflow.ai/ ERAN, the world's first scalable verifier for deep neural networks: https://github.com/eth-sri/eran VerX, the world's first fully automated verifier for smart contracts: https://verx.ch Securify, the first scalable security scanner for Ethereum smart contracts: https://securify.ch DeGuard, de-obfuscates Android binaries: http://apk-deguard.com SyNET, the first scalable network-wide configuration synthesis tool: https://synet.ethz.ch --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Petar on LinkedIn: https://www.linkedin.com/in/petartsankov/ Timestamps: [00:00] Petar's preferred coffee [00:29] Takeaways [03:15] Shout out to LatticeFlow for sponsoring this episode! [03:22] Please like, share, leave a review, and subscribe to our MLOps channels! [03:42] Expansion [05:16] Zurich ETH [07:06] AI Safety [09:24] Optimizing one metric, no fixed data sets [12:19] Trust life-changing issues [14:59] So much interest in GenAI [16:45] Explosion of GenAI Trust and Safety [21:14] Red Teaming [25:22] Trustworthy AI in Industry [27:43] DataOps Challenges [33:42] Trusting Third-Party Models [37:00] Testing Open Source Models [41:41] Specialized ML for Leasing [43:04] Regulation and Financial Incentives [45:30] Regulations Drive Innovation Balance [47:23] Regulations vs Certification: Voluntary Prove [52:24] Workflow Transparency: Trust & Efficiency [53:20] Engineers Balance Compliance Risks [54:53] Pushing Deep Learning Limits [57:31] Wrap up

Thank you to Zilliz our wonderful sponsors of this episode create some amazing stuff with Zilliz RAG - https://zilliz.com/vector-database-use-cases/llm-retrieval-augmented-generation

Sam Bean is a seasoned AI and machine learning expert, specializing in Large Language Models (LLMs) and search tech.

With a computer science background and a drive for innovation, Sam leads the team at Rewind AI in leveraging advanced tech to tackle complex challenges. MLOps podcast #217 with Sam Bean, Software Engineer (Applied AI) at Rewind.ai, The Real E2E RAG Stack. // Abstract What does a fully operational LLM + Search stack look like when you're running your own retrieval and inference infrastructure? What does the flywheel really mean for RAG applications? How do you maintain the quality of your responses? How do you prune/dedupe documents to maintain your document quality? // Bio Sam has been training, evaluating, and deploying production-grade inference solutions for language models for the past 2 years at You.com. Previous to that he built personalization algorithms at StockX. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://github.com/sam-h-bean/ REinforced Self Training (REST) - https://arxiv.org/pdf/2308.08998.pdf REST meets REACT - https://arxiv.org/pdf/2312.10003.pdf --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sam on LinkedIn: https://www.linkedin.com/in/samuel-h-bean/ Timestamps: [00:00] Sam's preferred coffee [00:11] Takeaways [03:52] A competitive coding pinball player [07:18] Sam's MLOps journey [10:33] Search Challenges with ML [15:04] Expensive evaluation [21:04] Labeling Parties Boost Data Quality [24:10] Zeno's Paradox of Motion [25:51] Sam's job at Rewind AI [29:35] Multimodal RAG [30:59 - 32:06] Zilliz Ad [32:07] University of Prague paper leak [36:38] Signals behind the scenes [39:28] Content Over Metadata Approach [43:22] Optionality around evaluation and search [48:35] Incremental Robustness Building [51:33] Solid Foundations for Success [53:42] Production RAGs [1:00:06] Thoughts on DSPy [1:05:40] Using DSPy in Production [1:08:26] Wrap up

Anass Bensrhir is the Associate Partner of McKinsey & Company Casablanca. Anu Arora is the Principal Data Engineering at McKinsey & Company.

Check out mckinsey.com/quantumblack MLOps podcast #214 with QuantumBlack AI by McKinsey's Principal Data Engineer, Anu Arora and Associate Partner, Anass Bensrhir, Managing Data for Effective GenAI Application brought to you by our Premium Brand Partner QuantumBlack AI by  @McKinsey . // Abstract Generative AI is poised to bring impact across all industries and business functions across industries While many companies pilot GenAI, only a few have deployed GenAI use cases, e.g., retailers are producing videos to answer common customer questions using ChatGPT. A majority of organizations are facing challenges to industrialize and scale, with data being one of the biggest inhibitors. Organizations need to strengthen their data foundations given that among leading organizations, 72% noted managing data among the top challenges preventing them from scaling impact. Furthermore, leaders noticed that +31% of their staff's time is spent on non-value-added tasks due to poor data quality and availability issues. // Bio Anu Arora Data architect(~12 years) and have experience in Big data technologies, API development, building scalable data pipelines including DevOps and DataOps, and building GenAI solutions. Anass Bensrhir Anass Leads QuantumBlack in Africa, he specializes in the Financial sector and helps organizations deliver successful large Data transformation programs. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.mckinsey.com/capabilities/quantumblack/how-we-help-clients --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Anu on LinkedIn: https://uk.linkedin.com/in/anu-arora-072012 Connect with Anass on LinkedIn: https://www.linkedin.com/in/abensrhir/ Timestamps: [00:00] Anass and Anu's preferred coffee [00:35] Takeaways [04:02] Please like, share, leave a review, and subscribe to our MLOps channels! [04:09] Huge shout out to our sponsor QuantumBlack! [04:29] Anu's tech background [06:31] Anass tech background [07:28] The landscape of data [10:37] Dealing with unstructured data [15:51] Data lakes and ETL processes [22:19] Data Engineers' Heavy Workload [29:49] Data privacy and PII in the new LLMs paradigm [36:13] Balancing LLM Adoption Risk [44:06] Effective LMS Implementation Strategy [49:00] Decisions: Create or Wait [50:39] Wrap up

Alex Volkov serves as the AI Evangelist with Weights & Biases, Host of ThursdAI, Founder and CEO Targum and AI Consultant GPU POOR Def. not an owl.

MLOps podcast #215 with Alex Volkov, AI Evangelist at Weights & Biases, Becoming an AI Evangelist. // Abstract Follow Alex's journey into the world of AI, from being interested in running his first AI models to founding an AI startup, running a successful weekly AI news podcast & newsletter, and landing a job with  @WeightsBiases . // Bio Alex Volkov is an AI Evangelist at Weights & Biases, celebrated for his expertise in clarifying the complexities of AI and advocating for its beneficial uses. He is the founder and host of ThursdAI, a weekly newsletter and podcast that explores the latest in AI, its practical applications, open-source and innovation. With a solid foundation as an AI startup founder and 20 years in full-stack software engineering, Alex offers a deep well of experience and insight into AI innovation. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Evaluation Survey: https://hq.yougot.us/primary/WebInterview/3AW6LW5D/Start Website: https://thursdai.news

Alex on X (+X spaces also are also there) - https://twitter.com/altryne/

ThursdAI podcast/newsletter - https://sub.thursdai.news

Denver local AI tinkerers meetup - https://denver-boulder.aitinkerers.org/

Weights & Biases Growth Team hack week review - https://www.youtube.com/watchInterview w/

Crew AI creator Joao Moura - https://sub.thursdai.news/p/jan14-sunday-special-deep-dives The Future of Search in the Era of Large Language Models // Saahil Jain // MLOps Podcast #150: https://youtu.be/hMoMvK89iogDSPy: Transforming Language Model Calls into Smart Pipelines // Omar Khattab // MLOps Podcast #194: https://youtu.be/NoaDWKHdkHg --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Alex on LinkedIn: https://www.linkedin.com/in/alex-volkov-/ Timestamps: [00:00] Alex's preferred beverage [00:17] Takeaways [03:19] Take the Evaluation Survey! [03:55] Alex's journey for the past 15 years [08:10] Career moves [15:15] Building communities [20:02] AI/MLOps Growth in COVID [27:23] Recent developments and insights [31:58 - 33:03] WandB Ad [33:54] Multimodal RAG and Lucid Dreaming [39:55] Evaluation Practices in MLOps [43:27] Evaluating AI models effectively [52:52] Embedding models and updates [56:14] AI model trade-offs [1:01:13] Optimizing LLM user experience [1:03:56] Perceived performance optimization [1:05:45] Agents' hype and reality [1:11:31] Exploring DSPy for evaluation [1:14:13] Wrap up

// Abstract

From startups achieving significant value with minor capabilities to AI revolutionizing sales calls and raising sales by 30%, we explore a series of interesting real-world use cases. Understanding the objectives and complexities of various industries, exploring the challenges of launching products, and highlighting the vital integration of the human touch with technology, this episode is a treasure trove of insights. // Bio Greg Kamradt - Moderator Greg has mentored thousands of developers and founders, empowering them to build AI-centric applications. By crafting tutorial-based content, Greg aims to guide everyone from seasoned builders to ambitious indie hackers. Some of his popular works: 'Introduction to LangChain Part 1, Part 2' (+145K views), and 'How To Question A Book' featuring Pinecone (+115K Views). Greg partners with companies during their product launches, feature enhancements, and funding rounds. His objective is to cultivate not just awareness, but also a practical understanding of how to optimally utilize a company's tools. He previously led Growth @ Salesforce for Sales & Service Clouds in addition to being early on at Digits, a FinTech Series-C company. Agnieszka Mikołajczyk-Bareła Senior AI Engineer@Chaptr working on LLMs. PhD, author of datasets, scientific papers, and publications with over 1800 citations, holding numerous scholarships and awards. Daily, she conducts her research on her grant "Detecting and overcoming bias in data with explainable artificial intelligence" Preludium, awarded by Polish National Centre. She is a co-organizer of PolEval2021 and PolEval 2022 tasks with punctuation prediction and restoration. She organizes and actively contributes to the scientific community in her free time: she managed and led the team during the HearAI project focused on modeling Sign Language. A former organizer and a team leader at the open-source project. As an ML Expert, she supports the project "Susana" designed to detect and read product expiry dates to help the Blind "see". Jason Liu Jason is a machine learning engineer and technical advisor. Arjun Kannan Arjun Kannan builds products, businesses, and teams. Currently building ResiDesk, bringing AI copilots to help real estate forecast renewals, reduce turnover, and hit their budget. Arjun built and led product and engineering functions at Climb Credit (serving 100k students, doubling loan growth for 3 years straight) and at BlackRock (creating $400mm in annual revenue), and helped build multiple startups and small companies before that. // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

Daniel Svonava is the Co-Founder of Superlinked. Daniel Svonava attended the Faculty of Informatics and Information Technologies, Slovak University of Technology. MLOps podcast #214 with Daniel Svonava, CEO & Co-founder at Superlinked, Information Retrieval & Relevance: Vector Embeddings for Semantic Search // Abstract In today's information-rich world, the ability to retrieve relevant information effectively is essential. This lecture explores the transformative power of vector embeddings, revolutionizing information retrieval by capturing semantic meaning and context. We'll delve into: - The fundamental concepts of vector embeddings and their role in semantic search - Techniques for creating meaningful vector representations of text and data - Algorithmic approaches for efficient vector similarity search and retrieval - Practical strategies for applying vector embeddings in information retrieval systems // Bio Daniel is an entrepreneurial technologist with a 20 year career starting with competitive programming and web development in highschool, algorithm research and Google & IBM Research internships during university, first entrepreneurial steps with his own computational photography startup and a 6 year tenure as a tech lead for ML infrastructure at YouTube Ads, where his ad performance forecasting engine powers the purchase of $10B of ads per year. Presently, Daniel is a co-founder of Superlinked.com - a ML infrastructure startup that makes it easier to build information-retrieval heavy systems - from Recommender Engines to Enterprise-focused LLM apps. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Daniel on LinkedIn: https://www.linkedin.com/in/svonava/?originalSubdomain=ch Timestamps: [00:00] Daniel's preferred coffee [00:13] Takeaways [04:59] Please like, share, leave a review, and subscribe to our MLOps channels! [05:22] Recommender system pivot insights [08:49] RaaS Challenges and solutions [10:23] Vector Compute vs Traditional Compute [13:20] String conversion challenges [17:18] Vector Computation in Recommender Systems [20:55] RAG system setup overview [26:00] ETL and Vector embeddings [31:04] Fine-tuning embedding models RAG [36:10] Flattening data for Vectors [37:18] Vector compute control insights [47:48] Vector Hub database comparison [51:22] Vector database partnership strategy [52:47] Vector computation in ML [55:43] Wrap up

Morgan McGuire has held a variety of roles in the past 13 years. In 2008, he completed a Research Internship at Queen Mary, University of London. Currently, he is the Head of Growth ML and Growth ML Engineer at Weights & Biases. Anish Shah has been working in the tech industry since 2015. In 2015, he was a Technical Support at Fox School of Business at Temple University. In 2021, he has been an MLOps Engineer - Growth and a Tier 2 Support Machine Learning Engineer at Weights & Biases. ______________________________________________ Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them? In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output, and debugging. You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches. Join the AI in Production Conference on February 22 here: https://home.mlops.community/home/events/ai-in-production-2024-02-15 ______________________________________________ MLOps podcast #213 with Weights and Biases' Growth Director, Morgan McGuire and MLE, Anish Shah, Evaluating and Integrating ML Models brought to you by our Premium Brand Partner  @WeightsBiases. // Abstract Anish Shah and Morgan McGuire share insights on their journey into ML, the exciting work they're doing at Weights and Biases, and their thoughts on MLOps. They discuss using large language models (LLMs) for translation, pre-written code, and internal support. They discuss the challenges of integrating LLMs into products, the need for real use cases, and maintaining credibility. They also touch on evaluating ML models collaboratively and the importance of continual improvement. They emphasize understanding retrieval and balancing novelty with precision. This episode provides a deep dive into Weights and Biases' work with LLMs and the future of ML evaluation in MLOps. It's a must-listen for anyone interested in LLMs and ML evaluation. // Bio Anish Shah Anish loves turning ML ideas into ML products. He started his career working with multiple Data Science teams within SAP, working with traditional ML, deep learning, and recommendation systems before landing at Weights & Biases. With the art of programming and a little magic, Anish crafts ML projects to help better serve our customers, turning “oh nos” to “a-ha”s! Morgan McGuire Morgan is a Growth Director and an ML Engineer at Weights & Biases. He has a background in NLP and previously worked at Facebook on the Safety team where he helped classify and flag potentially high-severity content for removal. // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI in Production Conference: https://home.mlops.community/home/events/ai-in-production-2024-02-15 Website: https://wandb.ai/ Prompt Templates the Song: https://www.youtube.com/watch?v=g6WT85gIsE8 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Morgan on LinkedIn: https://www.linkedin.com/in/morganmcg1/ Connect with Anish on LinkedIn: https://www.linkedin.com/in/anish-shah/

Alexandra Diem, PhD, has extensive experience in the field of AI, machine learning, and cloud analytics. Alexandra currently holds the position of Head of Cloud Analytics and MLOps at Gjensidige. Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them? In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output, and debugging. You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches. Join the AI in Production Conference on February 22 here: https://home.mlops.community/home/events/ai-in-production-2024-02-15 _____________________________________________________________ MLOps podcast #212 with Alexandra Diem, Head of Cloud Analytics & MLOps at Gjensidige, Data Governance and AI. // Abstract This recent session featuring the incredibly talented Alexandra Diem delves into the challenges of generative AI in sensitive data environments, the emergence of specialized chatbots, and data governance. Balancing high-tech projects with those offering significant business value, using agile methods, is also discussed. Alexandra's journey from academia to being a consultant in Norway is truly inspiring. The discussion explores the function of enabling and R&D in tech roles, the shift towards self-serve solutions, and the integration of AI into existing workflows. Stimulating conversations about future-oriented technologies married with sound data science and industry practices make this session a must-listen for anyone interested in machine learning operations! // Bio Former academic turned data scientist with a passion for data mesh architectures. 🔬 Background in applied mathematics and statistics, adept at leveraging data-driven insights to solve complex problems. Experienced in diverse domains spanning the private and public sectors. 🧠 Made significant contributions to research in physiological modeling, successfully debunking a leading biomedical hypothesis on Alzheimer's disease during my PhD. Developed innovative approaches to quantify blood supply to the heart. 💡 Solution-oriented thinker with a track record of efficiently tackling challenging problems and adapting to novel scenarios. ⚙️ Expertise: Data Science | Mathematical Modeling | Statistical Analysis | Problem Solving In my spare time, you'll find me exploring the great outdoors—whether it's pedaling through scenic landscapes on a bike or riding down the slopes on a pair of skis. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI in Production Conference: https://home.mlops.community/home/events/ai-in-production-2024-02-15 Website: https://github.com/alexdiem Talk "DevOps revolutionised software engineering, it's time to revolutionise data" https://vimeo.com/861721829 from JavaZone 2023 Zilliz Cloud: https://zilliz.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Alexandra on LinkedIn: https://www.linkedin.com/in/dralexdiem/

Aayush Mudgal is a Senior Machine Learning Engineer at Pinterest, currently leading the efforts around Privacy-Aware Conversion Modeling. Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them? In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output, and debugging. You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches. Join the AI in Production Conference on February 15 and 22 here: https://home.mlops.community/home/events/ai-in-production-2024-02-15 ________________________________________________________________________________________ MLOps podcast #211 with Aayush Mudgal, Senior Machine Learning Engineer at Pinterest, Ads Ranking Evolution at Pinterest. // Abstract Listen to the lessons from the journey of scaling ads ranking at Pinterest using innovative machine learning algorithms and innovation in the ML platform. Learn how they transitioned from traditional logistic regressions to deep learning-based transformer models, incorporating sequential signals, multi-task learning, and transfer learning. Discover the hurdles Pinterest overcame and the insights they gained in this talk, as Aayush shares the transformation of ads ranking at Pinterest and the lessons learned along the way. Discover how ML Platform evolution is crucial for algorithmic advancements. // Bio Aayush Mudgal is a Senior Machine Learning Engineer at Pinterest, currently leading the efforts around Privacy-Aware Conversion Modeling. He has a successful track record of starting and executing 0 to 1 projects, including conversion optimization, video ads ranking, landing page optimization, and evolving the ads ranking from GBDT to DNN stack. His expertise is in large-scale recommendation systems, personalization, and ads marketplaces. Before entering the industry, Aayush conducted research on intelligent tutoring systems, developing data-driven feedback to aid students in learning computer programming. He holds a Master's in Computer Science from Columbia University and a Bachelor of Technology in Computer Science from the Indian Institute of Technology Kanpur. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.youtube.com/watch?v=MZVIxtsGzBg https://www.youtube.com/watch?v=ffpPUr8Hg6U --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Aayush on LinkedIn: https://www.linkedin.com/in/aayushmudgal/

Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them? In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output, and debugging. You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches. Join the AI in Production Conference on February 15 and 22 here: https://home.mlops.community/home/events/ai-in-production-2024-02-15 ________________________________________________________________________________________
Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, a pioneer and early leader in machine learning (ML) observability. MLOps podcast #210 with Aparna Dhinakaran, Co-Founder and Chief Product Officer of Arize AI, LLM Evaluation with Arize AI's Aparna Dhinakaran. // Abstract Dive into the complexities of Language Model (LLM) evaluation, the role of the Phoenix evaluations library, and the importance of highly customized evaluations in software application. The discourse delves into the nuances of fine-tuning in AI, the debate between the use of open-source versus private models, and the urgency of getting models into production for early identification of bottlenecks. Then examine the relevance of retrieved information, output legitimacy, and the operational advantages of Phoenix in supporting LLM evaluations. // Bio Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, a pioneer and early leader in AI observability and LLM evaluation. A frequent speaker at top conferences and thought leader in the space, Dhinakaran is a Forbes 30 Under 30 honoree. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she built several core ML Infrastructure platforms, including Michelangelo. She has a bachelor’s from Berkeley's Electrical Engineering and Computer Science program, where she published research with Berkeley's AI Research group. She is on a leave of absence from the Computer Vision Ph.D. program at Cornell University. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Arize-Phoenix: https://phoenix.arize.com/ Phoenix LLM task eval library: https://docs.arize.com/phoenix/llm-evals/running-pre-tested-evals Aparna's recent piece on LLM evaluation: https://arize.com/blog-course/llm-evaluation-the-definitive-guide/ Thread on the difference between model and task LLM evals: https://twitter.com/aparnadhinak/status/1752763354320404488 Research thread on why numeric score evals are broken: https://twitter.com/aparnadhinak/status/1748368364395721128 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Aparna on LinkedIn: https://www.linkedin.com/in/aparnadhinakaran/

Matt Bleifer is a Group Product Manager at Tecton, where he focuses on the core product experience such as building, testing, and productionizing feature pipelines as scale. Michael Eastham works as a Chief Architect at Tecton, which is a Business Intelligence (BI) Software company with an estimated 100 employees Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them? In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output, and debugging. You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches. Join the AI in Production Conference on February 15 and 22 here: https://home.mlops.community/home/events/ai-in-production-2024-02-15 ________________________________________________________________________________________ MLOps podcast #209 with Tecton's Group Product Manager, Matt Bleifer and Chief Architect, Mike Eastham, Powering MLOps: The Story of Tecton's Rift brought to us by our Premium Brand Partner,  @tecton8241 . // Abstract Explore the intricacies of feature platforms and their integration in the data realm. Compare traditional predictive machine learning with the integration of Linguistic Model Systems into software applications. Get a glimpse of Rift, a product enhancing data processing with smooth compatibility with various technologies. Join in on the journey of developing Rift, and making Tecton user-friendly, and enjoy Matt's insights and contributions. Wrap it up with lighthearted talks on future collaborations, music, and a touch of nostalgia. // Bio Matt Bleifer Matt Bleifer is a Group Product Manager and an early employee at Tecton. He focuses on core product experiences such as building, testing, and productionizing feature pipelines at scale. Before joining Tecton, he was a Product Manager for Machine Learning at both Twitter and Workday, totaling nearly a decade of working on machine learning platforms. Matt has a Bachelor’s Degree in Computer Science from California Polytechnic State University, San Luis Obispo. Michael Eastham Michael Eastham is the Chief at Tecton. Previously, he was a software engineer at Google, working on Web Search. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.tecton.ai/Rift Article: https://www.tecton.ai/blog/unlocking-real-time-ai-for-everyone-with-tecton/ Rift: https://resources.tecton.ai/riftBig Data is Dead blog: https://motherduck.com/blog/big-data-is-dead/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Matt on LinkedIn: https://www.linkedin.com/in/mattbleifer/Connect with Mike on LinkedIn: https://www.linkedin.com/in/mikeeastham/ Timestamps: [00:00] AI in Production Conference [02:13] Matt & Mike's preferred coffee [02:37] Takeaways [04:50] Matt & Mike's Tecton titles [06:49] Matt's background in tech [09:49] Mike's background in tech [12:53] Tecton refresher [18:23] Feature store to Feature platform [21:06] Current evolution of Tecton [24:41] The understatement [26:12] Duck DB Con [27:54] Rift [30:10] Kafka Flink [33:36] What is large in aggregations? [38:09] Big Data is Dead! [41:14] Principles of creating Rift [45:54] The battle between Simplicity and Flexibility [47:28] Is he serious? Segment [50:54] Can you get any more hype Segment [57:10] What are you excited about? [1:02:45] Wrap up

Join our virtual conference 'AI in Production'

Transform faster. Innovate smarter. Anticipate the future. At QuantumBlack, we unlock the power of artificial intelligence (AI) to help organizations reinvent themselves from the ground up—and accelerate sustainable and inclusive growth.

MLOps Coffee Sessions Special episode with QuantumBlack, AI by McKinsey, GenAI Buy vs Build, Commercial vs Open Source, fueled by our Premium Brand Partner, QuantumBlack, AI by McKinsey. // Abstract Do you build or buy? Check the QuantumBlack team discussing the different sides of buying vs building your own GenAI solution. Let's look at the trade-offs companies need to make - including some of the considerations of using black box solutions that do not provide transparency on what data sources were used. Whether you are a business leader or a developer exploring the space of GenAI, this talk provides you with valuable insights to prepare you for how you can be more informed and prepared for navigating this fast-moving space. // Bio Ilona Logvinova Ilona Logvinova is the Head of Innovation for McKinsey Legal, working across the legal department to identify, lead, and implement cross-cutting and impactful innovation initiatives, covering legal technologies and reimagination of the profession initiatives. At McKinsey Ilona is also Managing Counsel for McKinsey Digital, working closely with emerging technologies across use cases and industries. Mohamed Abusaid Am Mohamed, a tech enthusiast, hacker, avid traveler, and foodie all rolled into one individual. Built his first website when he was 9 and fell in love with computers and the internet ever since. Graduated with a computer science from university although dabbled in electrical, electronic, and network engineering before that. When he's not reading up on the latest tech conversations and products on Hacker News, Mohamed spends his time traveling to new destinations and exploring their cuisine and culture. Mohamed works with different companies helping them tackle challenges in developing, deploying, and scaling their analytics to reach its potential. Some topics he's enthusiastic about include MLOps, DataOps, GenerativeAI, Product thinking, and building cross-functional teams to deliver user-first products. Nayur Khan Nayur is a partner within McKinsey and part of the QuantumBlack, AI by McKinsey leadership team. He predominantly focuses on helping organizations build capabilities to industrialize and scale artificial intelligence (AI), including the newer Generative AI. He helps companies navigate innovations, technologies, processes, and digital skills as needed to run at scale. He is a keynote speaker and is recognized in the DataIQ 100 - a list of the top 100 influential people in data. Nayur also leads the firm’s diversity and inclusion efforts within QuantumBlack to promote a more equitable environment for all. He speaks with organizations on the importance of diversity and diverse team building—especially when working with data and building AI. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ilona on LinkedIn: www.linkedin.com/in/ilonalogvinova Connect with Mo on LinkedIn: https://www.linkedin.com/in/mabusaid/ Connect with Nayur on LinkedIn: https://www.linkedin.com/in/nayur/

Jon Cooke is the owner/founder of Dataception a Data, Analytics, and Data Product company, and the creator of the Data Product Pyramid, an adaptive Data Product operating model.

MLOps podcast #208 with Jon Cooke, CTO of Dataception, Micro Graph Transformer - Specialist Small Language Models Using Graphs to Accelerate Data Product Eco-systems. // Abstract Specialist deconstructed Encoder/Decoder Transformers along with data product management and tech to vastly accelerate prototyping, building, and deploying business-facing data products at high speed and low cost. // Bio Jon is a 20-year veteran in Data, Analytics, and AI and is a Data product specialist. After many times seeing the massive time, friction, failures, and costs typically associated with data and analytics initiatives, Jon founded Dataception. Its mission is to use tech to eliminate the data grunt and work together with data product management and AI to build and iterate sophisticated, business-facing analytics in real-time in front of the business. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI in Production Conference: https://home.mlops.community/public/events/ai-in-production-2024-02-15 Website: www.dataception.com https://www.linkedin.com/events/generativeai-dataproductsandbus7114951387100184576/theater/ https://www.linkedin.com/events/12thevalueofadataproductmanagem7110920848416366594/comments/ https://www.linkedin.com/events/howtoactuallyusedataproductstod7113570339535638528/theater/Building Better Data Teams // Leanne Fitzpatrick // Coffee Sessions #113: https://www.youtube.com/watch?v=JxVS3-4wyKc --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jon on LinkedIn: https://www.linkedin.com/in/jon-cooke-096bb0/

Timestamps: [00:00] AI in Production Conference teaser [02:12] Jon's preferred coffee [02:24] Takeaways [03:48] Please like, share, and subscribe to our MLOps channels! [04:02] Backpacking, traveling, and almost cast for Lord of the Rings [06:40] Jon's tech background [11:07] Dataception [15:05] Data Challenges: Delays & Causes [16:46] Data Virtualization for Agility [19:47] Large Company Change Challenges [21:28] Sales Tools Migration Challenges [24:44] Data and ML Integration [28:13] Data Roles Evolution [32:20] Tech for Prototyping Acceleration [35:22] LLM Enables Natural Language Data Analytics [36:36] Ensuring Reliable AI Information [38:20] Proxy Routing and Intelligent Agents [42:41] Human API for Data [46:49] Engineer Success with Growth [48:15] Tech CEO Balancing Act [53:59] Iterative Development for Product-Market Fit [56:16] Wrap up

Jake Watson is the writer of thedataplatform.substack.com⁠ and Principal Data Engineer at The Oakland Group.

MLOps podcast #207 with Jake Watson, Principal Data Engineer at The Oakland Group, How Data Platforms Affect ML & AI. // Abstract I’ve always told my clients and colleagues that traditional rule-based software is difficult, but software containing Artificial Intelligence (AI) and/or Machine Learning (ML)* is even more difficult, sometimes impossible. Why is this the case? Well, software is difficult because it’s like flying a plane while building it at the same time, but because AI and ML make rules on the fly based on various factors like training data, it’s like trying to build a plane in flight, but some parts of the plane will be designed by a machine, and you have little idea what that is going to look like till the machine finishes. This double goes for more cutting-edge AI models like GPT, where only the creators of the software have a vague idea of what it will output. This makes software with AI / ML more of a scientific experiment than engineering, which is going to make your project manager lose their mind when you have little idea how long a task is going to take. But what will make everyone’s lives easier is having solid data foundations to work from. Learn to walk before running. // Bio Jake has been working in data as an Analyst, Engineer, and/or Architect for over 10 years. Started as an analyst in the UK National Health Service converting spreadsheets to databases tracking surgical instruments. Then continued as an analyst at a consultancy (Capita) reporting on employee engagement in the NHS and dozens of UK Universities. There Jake moved reporting from Excel and Access to SQL Server, Python with frontend websites in d3.js. At Oakland Group, a data consultancy, Jake worked as a Cloud Engineer, Data Engineer, Tech Lead, and Architect depending on the project for dozens of clients both big and small (mostly big). Jake has also developed and productionised ML solutions as well in the NLP and classification space. Jake has experience in building Data Platforms in Azure, AWS, and GCP (though mostly in Azure and AWS) using Infrastructure as Code and DevOps/DataOps/MLOps. In the last year, Jake has been writing articles and newsletters for my blog, including a guide on how to build a data platform: https://thedataplatform.substack.com/p/how-to-build-a-data-platform // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://thedataplatform.substack.com/ How Data Platform Foundations Impact AI and ML Applications blog: https://thedataplatform.substack.com/p/issue-29-how-data-platform-foundations AI in Production Conference: https://home.mlops.community/public/events/ai-in-production-2024-02-15 How to Build a Data Platform blog: https://thedataplatform.substack.com/p/how-to-build-a-data-platform --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jake on LinkedIn: https://www.linkedin.com/in/jake-watson-data/ Timestamps: [00:00] Jake's preferred coffee [00:26] AI in Production Conference teaser [02:38] Takeaways [04:00] Please like, share, and subscribe to our MLOps channels! [04:17] Data Engineer's Crucial Role [05:44] Jake's background [06:44] Data Platform Foundations blog [10:34] Data mesh organizational side of things [17:58] Importance of data modeling [20:13] Dealing with the sprawl [22:03] Data quality [23:59] Data hierarchy on building a platform [29:34] ML Platform Team Structure [31:47] Don't reinvent the wheel [34:04] Data pipelines synergy [37:31] Wrap up

Yujian is working as a Developer Advocate at Zilliz, where they develop and write tutorials for proof of concepts for large language model applications. They also give talks on vector databases, LLM Apps, semantic search, and tangential spaces.

MLOps podcast #206 with Yujian Tang, Developer Advocate at Zilliz, RAG Has Been Oversimplified, brought to us by our Premium Brand Partner, Zilliz // Abstract In the world of development, Retrieval Augmented Generation (RAG) has often been oversimplified. Despite the industry's push, the practical application of RAG reveals complexities beyond its apparent simplicity. This talk delves into the nuanced challenges and considerations developers encounter when working with RAG, providing a candid exploration of the intricacies often overlooked in the broader narrative. // Bio Yujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published to conferences including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: zilliz.com --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Yujian on LinkedIn: linkedin.com/in/yujiantang Timestamps: [00:00] Yujian's preferred coffee [00:17] Takeaways [02:42] Please like, share, and subscribe to our MLOps channels! [02:55] The hero of the LLM space [05:42] Embeddings into Vector databases [09:15] What is large and what is small LLM consensus [10:10] QA Bot behind the scenes [13:59] Fun fact getting more context [17:05] RAGs eliminate the ability of LLMs to hallucinate [18:50] Critical part of the rag stack [19:57] Building citations [20:48] Difference between context and relevance [26:11] Missing prompt tooling [27:46] Similarity search [29:54] RAG Optimization [33:03] Interacting with LLMs and tradeoffs [35:22] RAGs not suited for [39:33] Fashion App [42:43] Multimodel Rags vs LLM RAGs [44:18] Multimodel use cases [46:50] Video citations [47:31] Wrap up

Jonathan Frankle works as Chief Scientist (Neural Networks) at MosaicML (recently acquired by Databricks), a startup dedicated to making it easy and cost-effective for anyone to train large-scale, state-of-the-art neural networks. He leads the research team. MLOps podcast #205 with Jonathan Frankle, Chief Scientist (Neural Networks) at Databricks, The Myth of AI Breakthroughs, co-hosted by Denny Lee, brought to us by our Premium Brand Partner, Databricks. // Abstract Jonathan takes us behind the scenes of the rigorous work they undertake to test new knowledge in AI and to create effective and efficient model training tools. With a knack for cutting through the hype, Jonathan focuses on the realities and usefulness of AI and its application. We delve into issues such as face recognition systems, the 'lottery ticket hypothesis,' and robust decision-making protocols for training models. Our discussion extends into Jonathan's interesting move into the world of law as an adjunct professor, the need for healthy scientific discourse, his experience with GPUs, and the amusing claim of a revolutionary algorithm called Qstar. // Bio Jonathan Frankle is Chief Scientist (Neural Networks) at Databricks, where he leads the research team toward the goal of developing more efficient algorithms for training neural networks. He arrived via Databricks’ $1.3B acquisition of MosaicML as part of the founding team. He recently completed his PhD at MIT, where he empirically studied deep learning with Prof. Michael Carbin, specifically the properties of sparse networks that allow them to train effectively (his "Lottery Ticket Hypothesis" - ICLR 2019 Best Paper). In addition to his technical work, he is actively involved in policymaking around challenges related to machine learning. He earned his BSE and MSE in computer science at Princeton and has previously spent time at Google Brain and Facebook AI Research as an intern and Georgetown Law as an Adjunct Professor of Law. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: www.jfrankle.com Facial recognition: perpetuallineup.orgThe Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networksby Jonathan Frankle and Michael Carbin paper: https://arxiv.org/abs/1803.03635 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Denny on LinkedIn: https://linkedin.com/in/dennyglee Connect with Jonathan on LinkedIn: https://www.linkedin.com/in/jfrankle/ Timestamps: [00:00] Jonathan's preferred coffee [01:16] Takeaways [07:19] LM Avalanche Panel Surprise [10:07] Adjunct Professor of Law [12:59] Low facial recognition accuracy [14:22] Automated decision making human in the loop argument [16:09] Control vs. Outsourcing Concerns [18:02] perpetuallineup.org [23:41] Face Recognition Challenges [26:18] The lottery ticket hypothesis [29:20] Mosaic Role: Model Expertise [31:40] Expertise Integration in Training [38:19] SLURM opinions [41:30] GPU Affinity [45:04] Breakthroughs with QStar [49:52] Deciphering the noise advice [53:07] Real Conversations [55:47] How to cut through the noise [1:00:12] Research Iterations and Timelines [1:02:30] User Interests, Model Limits [1:06:18] Debugability [1:08:00] Wrap up

Patrick Barker is the Founder / CTO of Kentauros AI.

Farhood Etaati is a Software Engineer at Yektanet. MLOps podcast #204 with Patrick Barker, CTO of Kentauros AI and Farhood Etaati, MLOps/Platform Team Lead at AIMedic, MLOps at the Crossroads. // Abstract MLOps is at a crossroads. The ever-increasing excitement for LLMs' ability to solve some interesting real-world problems has made many people interested in applying these models in new applications which comes with its own challenges, that have upstarted the term "LLMLOps". But how much of those challenges are not a newer representation of what older-gen ML models had to deal with in the production, and the question arises whether developing "new" specialized tools to address these applications actually provides any substantial value for the sustainability of the field in general. Tools are coming and going at a rate that makes many technical people skeptical of adopting newer tools. What can we do as a community to alleviate these issues? Why OSS MLOps is lacking behind and how VC money is contributing to that? // Bio Farhood Etaati MLOps engineer at AIMedic. Studied EE at Uni of Tehran, started out as a data scientist, and pivoted to software engineering. Currently working on on-premise MLOps platform development suitable for Iran's infrastructure. Patrick Barker When Patrick is not occupied with building his AI company, he enjoys spending time with his wonderful kids or exploring the hills of Boulder. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Websites: https://github.com/pbarker

--------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Patrick on LinkedIn: https://www.linkedin.com/in/patrickbarkerco/ Connect with Farhood on LinkedIn: www.linkedin.com/in/farhood-etaati Timestamps: [00:00] Farhood's and Patrick's preferred coffee [01:13] Takeaways [04:00] Please like, share, and subscribe to our MLOps channels! [05:26] Strong feelings [10:21] MLOps vs DevOps Challenges [13:44] Medical setting, ML tools, NLP, model building [16:23] MLOps vs Data Engineering [20:45] MLOps Boosts LLM Development [23:54] Longtail Use Cases [31:00] Tech Roles Distinctions [34:42] Did He Say That? [37:04] Fine-tuning AI Models [38:57] ML 2.0 Advancements Explained [41:11] Generative AI in MLOps [45:04] ML Reproducibility Challenges [48:03] Wrap up

Aleksa Gordić is an ex-Google DeepMind / Microsoft ML engineer currently working on non-English LLMs at OrtusAI, open-sourcing Meta's NLLB (no language left behind) project and YugoGPT.

MLOps podcast #203 with Aleksa Gordić, Founder of OrtusAI, Pioneering AI Models for Regional Languages. // Abstract Dive deep into Aleksa's work with the YugoGPT, a language model serving Serbian, Croatian, Bosnian, and Montenegrin dialects - emphasizing the need for multilingual AI developments. Explore the unique language dynamics in the Balkans and Eastern Europe, the potential business opportunities around multilingual models, and the challenges in deploying large language models. Aleksa shares his experience with vision and image models, his collaborations with key tech players, and his use of advanced technologies. Hear about Aleksa Gordić's journey of being active and visible in the tech community and his insights into the world of machine learning and AI. Prepare to have your thinking challenged and horizons widened as we converse about the intriguing and complex world of MLOps. // Bio Working on non-English LLMs at OrtusAI, open-sourcing Meta's NLLB (no language left behind) project. Worked at DeepMind on the Flamingo project as a research engineer. Worked at Microsoft on the HoloLens 2 project & next-gen mixed reality glasses. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://gordicaleksa.com/ https://github.com/gordicaleksa - I build stuff :) https://discord.com/invite/peBrCpheKE - active AI Discord server (~6000) I bring the best AI researchers in the world to give talks (James Betker DALL-E 3 author, Tri Dao (Flash Attention), etc.) https://gordicaleksa.medium.com/how-i-got-a-job-at-deepmind-as-a-research-engineer-without-a-machine-learning-degree-1a45f2a781de - how I landed a job at DeepMind (and a couple more potentially interesting writings) Aleksa Gordić The AI Epiphany Youtube Channel: https://www.youtube.com/channel/UCj8shE7aIn4Yawwbo2FceCQ/videos W&B AI Academy: http://wandb.me/mlops_com_llm_course ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Aleksa on LinkedIn: https://www.linkedin.com/in/aleksagordic/ Timestamps: [00:00] Aleksa's preferred coffee [00:17] Takeaways [02:51] Humming the GPU's [06:23] Built Chrome extension for communicating with videos [08:04] Rig Doubles Throughput Time [09:32] Vector databases advise [10:38] Learning from experts, connecting, and gathering insights. [13:47] Zero to Hero for MLOps [15:37] Serendipitous moments [17:52] Depth Over Breaking News [19:50] Trust in GPT Content [22:22] Exam Challenges and AI [26:53] YugoGPT [31:41] WandB Ad [33:33] Linguistic Mysteries [34:52] No Language Left Behind project (NLLB project) [36:53] YugoGPT Development Overview [37:49] NLLB vs YugoGPT [39:35] Yugo GPT parameters [41:16] Opportunities for unsupported languages [43:08] Diffusion model [44:39] Generative AI with image generation models [47:45] AI Challenges and Excitement [50:32] Challenges in different alphabet characters [52:10] Need a co-founder [56:05] Career transition and entrepreneurial mindset [1:00:20] Big Tech salary misconceptions [1:03:02] Inspiring wrap up

Prof. Dr. Hannes Mühleisen is a creator of the DuckDB database management system and Co-founder and CEO of DuckDB Labs. Jordan is co-founder and chief duck-herder at MotherDuck, a startup building a serverless analytics platform based on DuckDB.

MLOps podcast #202 with Hannes Mühleisen, Co-Founder & CEO of DuckDB Labs and Jordan Tigani, Chief Duck-Herder at MotherDuck, Small Data, Big Impact: The Story Behind DuckDB. // Abstract Navigate the intricacies of data management with Jordan Tagani and Hannes Mühleisen, the creative geniuses behind DuckDB and MotherDuck. This deep dive unravels the game-changing principles behind DuckDB's creation, tackling the prevailing wisdom to passionately fill the gap for smaller data set management. Let's also discover MotherDuck's unique focus on providing an unprecedented developer experience and its innovative edge in visualization and data delivery. This episode is teeming with enlightening discussions about managing community feedback, funding, and future possibilities that should not be missed for any tech enthusiasts and data management practitioners. // Bio Hannes Mühleisen Prof. Dr. Hannes Mühleisen is a creator of the DuckDB database management system and Co-founder and CEO of DuckDB Labs, a consulting company providing services around DuckDB. Hannes is also Professor of Data Engineering at Radboud Universiteit Nijmegen. His' main interest is analytical data management systems. Jordan Tigani Jordan is co-founder and chief duck-herder at MotherDuck, a startup building a serverless analytics platform based on DuckDB. He spent a decade working on Google BigQuery, as a founding engineer, book author, engineering leader, and product leader. More recently, as SingleStore’s Chief Product Officer, Jordan helped them build a cloud-native SaaS business. Jordan has also worked at Microsoft Research, the Windows Kernel team, and at a handful of star-crossed startups. His biggest claim to fame is predicting world cup matches using machine learning with a better record than Paul the Octopus. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Websites: https://duckdb.org/ https://motherduck.com/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hfmuehleisen/ Connect with Jordan on LinkedIn: https://www.linkedin.com/in/jordantigani/ Timestamps: [00:00] Hannes and Jordan's preferred coffee [01:30] Takeaways [03:43] Swaggers in the house! [07:13] Duck DB's inception [09:38] Jordan's background [12:28] Simplify Developer Experience [17:54] Big Data Shift [26:01] Creation of MotherDuck [30:58] Duck DB and MotherDuck Partnership [31:57] Incentive Alignment Concerns [37:46] Building an incredible developer experience [43:38] User Testing Lab [47:18] Setting a higher standard [49:22] The moments before the moment [52:18] Gathering feedback and talking to the community [54:30] MotherDuck Features [1:00:19] Cloud Innovation for MotherDuck [1:02:41] ML Engineers and DuckDB [1:08:03] Wrap up

Paco Nathan is the Managing Partner at Derwen, Inc., and author of Latent Space, along with other books, plus popular videos and tutorials about machine learning, natural language, graph technologies, and related topics.

MLOps podcast #201 with Paco Nathan, Managing Partner at Derwen, Inc., Language, Graphs, and AI in Industry. // Abstract Let's talk about key findings from these conferences, specifically summarizing teams that have ROI on machine learning in production: what are the things in common they're doing, and what are the most important caveats they urge other teams to consider when getting started? Because these key takeaways aren't found in the current AI news cycle. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI Conference: https://aiconference.com/ K1st World: https://www.k1st.world/ Corunna Innovation Summit: https://corunna.dataspartan.com/ "Cloud Computing on Amazon AWS EC2" UC Berkeley EECS guest lecture (2009) https://vimeo.com/manage/videos/3616394 "Hardware - Software - Process: Data Science in a Post-Moore’s Law World" https://www.nvidia.com/en-us/ai-data-science/resources/hardware-software-process-book/ “LLMs in Production: Learning from Experience” by Waleed Kadous @ Anyscale https://www.youtube.com/watch?v=xa7k9MUeIdk "Supercharging Industrial Operations with Problem-Solving GenAI & Domain Knowledge" by Christopher Nguyen @ Aitomatic https://www.k1st.world/2023-program/supercharging-industrial-operations-with-problem-solving-genai-domain-knowledge “The Next Million AI Systems” by Mark Huang @ Gradient: https://www.youtube.com/watch?v=lA0Npe4PqFw "AI in a Box" by Useful Sensors https://usefulsensors.com/#products "Opportunities in AI - 2023" by Andrew Ng https://www.youtube.com/watch?v=5p248yoa3oE "Advancing the Marine Industry Through the Harmony of Fishermen Knowledge and Al" by Akinori Kasai @ Furuno https://www.k1st.world/2023-program/advancing-the-marine-industry-through-the-harmony-of-fishermen-knowledge-and-al Macy conferences (1941-1960) https://en.wikipedia.org/wiki/Macy_conferences https://www.asc-cybernetics.org/foundations/history/MacySummary.htm https://press.uchicago.edu/ucp/books/book/distributed/C/bo23348570.html second-order cybernetics https://pangaro.com/designconversation/wp-content/uploads/dubberly-pangaro-chk-journal-2015.pdf https://en.wikipedia.org/wiki/Second-order_cybernetics Project Cybersyn https://jacobin.com/2015/04/allende-chile-beer-medina-cybersyn/ https://thereader.mitpress.mit.edu/project-cybersyn-chiles-radical-experiment-in-cybernetic-socialism/ https://99percentinvisible.org/episode/project-cybersyn/ https://medium.com/@rjog/project-cybersyn-an-early-attempt-at-iot-governance-and-how-we-can-apply-its-learnings-5164be850413 https://www.sustema.com/post/project-cybersyn-how-a-chilean-government-almost-controlled-the-economy-from-a-control-room https://transform-social.org/en/texts/cybersyn/ Humberto Maturana, Francisco Varela: Autopoeisis "De Maquinas y Seres Vivos" "Everything said is said by an observer" https://proyectos.yura.website/wp-content/uploads/2021/06/de_maquinas_y_seres_vivos_-_maturana.pdf https://en.wikipedia.org/wiki/Autopoiesis_and_Cognition:_The_Realization_of_the_Living Fernando Flores (led Project Cybersyn, imprisoned, later worked with Prof. Terry Winograd @ Stanford, the grad advisor for what became Google) https://lorenabarba.com/gallery/prof-barba-gave-keynote-at-pycon-2016/ https://conversationsforaction.com/fernando-flores "Navigating the Risk Landscape: A Deep Dive into Generative AI" by Ben Lorica and Andrew Burt https://thedataexchange.media/mitigating-generative-ai-risks/ "SpanMarker" by Tom Aarsen @ Hugging Face https://tomaarsen.github.io/SpanMarkerNER/ Examples of "the math catching up with the machine learning": Guy Van den Broeck @ UCLA https://web.cs.ucla.edu/~guyvdb/talks/ Charles Martin @ Calculations Consulting https://weightwatcher.ai/

Mihail Eric is an engineer, researcher, and educator who has helped start teams at innovative organizations such as Amazon Alexa and RideOS.

Mihail is a cofounder of Storia AI where they build an AI-powered creative assistant for fast and delightful image and video generation.

MLOps podcast #200 with Mihail Eric, Co-founder of Storia AI, Founding, Funding, and the Future of MLOps. // Abstract Demetrios and Mihail journey deep into the significance of human sentiment in an increasingly AI-driven era, the perils and promises of conversational AI, and the evolution and impact of image generation models. Delve into the world of MLOps versus LLMOps, offering clarifying perspectives on how the core concerns and technology persist, even amidst an evolving tech landscape with new buzzwords making waves. Mihail generously provides an inside look at his AI tool and its wide range of applications across various industries, offering insights on interesting niche-specific verticals and unexpected use cases. // Bio Mihail is a co-CEO of Storia AI, an early-stage startup building an AI-powered creative assistant for video production. He has over a decade of experience researching and engineering AI systems at scale. Previously he built the first deep-learning dialogue systems at the Stanford NLP group. He was also a founding member of Amazon Alexa’s first special projects team where he built the organization’s earliest large language models. Mihail is a serial entrepreneur who previously founded Confetti AI, a machine-learning education company that he led until its acquisition in 2022. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: www.mihaileric.com

https://www.storia.ai/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/ Timestamps: [00:00] Spotify playlist [02:08] How to live a longer healthier life [03:43] Absurd sweater collection [06:03] The catch-up episode [08:10] MLOps versus LLMOps [13:27] AI apps mixing visuals and text [16:00] Founder dating [16:54] Stable diffusion and difficulties with Mid Journey [23:12] Stripe developer experience [25:04] APIs and Model Providers [27:33] Host stable diffusion on AWS [34:07] AI Creativity: Prompt Experimentation [35:45] AI Challenges and Solutions [39:51] AI Hype Frustration [44:31] AI Impact on Hollywood [48:11] AI Impact on Filmmaking [51:48] Generalizable Tool for Verticals [52:49] MLOps versus LLMOps [56:42] Wrap up

Nathan Ryan Frank is the Machine Learning Operations and platform Director of Grainger. Former Astrophysicist turned data scientist and machine learning engineer with a proven history of delivering results into production across a wide variety of domains while leading projects with international, cross-functional teams. MLOps podcast #199 with Nathan Ryan Frank, Director, Machine Learning Platform & Operations at WW Grainger, Challenges Operationalizing Machine Learning (And Some Solutions). // Abstract This talk details some common challenges and pitfalls when attempting to operationalize machine learning systems and discusses some simple solutions. We dive into the machine learning development workflow and cover topics such as team dynamics, communication issues between roles that don't share a common language, and approaching MLOps from an SRE/DevOps perspective. Similarly, the talk highlights some challenges unique to operationalizing machine learning, drawing distinctions where necessary to highlight a large amount of similarity. Finally, the talk offers some simple and practical guidance for those new to MLOps who want to understand where to start and how to adopt best practices in an evolving field. // Bio Nathan Frank is currently the Director of Machine Learning Platform and Operations at Grainger where he is building a team to support the Technology Group's expanding machine learning efforts. Prior to joining Grainger, Nathan led machine learning engineering efforts at Strong Analytics, a boutique data science and machine learning consulting firm, as well as machine learning platform and development teams at Stats Perform, a leader in sports data and technology. Nathan holds bachelor's and master's degrees in Astrophysics from UC - Santa Cruz and UNC-Chapel Hill, respectively. When not building machine learning systems, Nathan spends as much time as possible with his favorite person in the world, his wife, as well as their four kids and two dogs, and enjoys getting outside to hike or garden and baking bread. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://nrfrank.github.io/ Bisi: https://bisi.gitbook.io/bisi/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Nathan on LinkedIn: https://www.linkedin.com/in/nrfrank Timestamps: [00:00] Nathan's preferred coffee [00:40] Takeaways [02:00] Please leave a review in our comment sections! Please like, share, and subscribe to our MLOps channels! [03:00] Telescope for gamma-ray burst [07:31] Transition into ML [11:23] Stats-heavy US sports commentary [14:25] Building machine learning systems approach [20:02] ML Workflow Must-Haves [26:50] Love for tests [33:10] Test Writing Importance [34:37] Bridging Stakeholder Language Gap [43:04] Shared Language, Team Collaboration [47:28] Rapid fire questions [51:20] Wrap up

Nick Hasty is the Director of Product, Discovery & Machine Learning at Giphy, an animated-gif search engine that allows users to search, share, and discover GIFs.

MLOps podcast #198, Inferring Creativity. // Abstract Generative AI models have captured our imaginations with their ability to produce new "creative" works such as visually striking images, poems, stories, etc, and their outputs often rival or excel what most humans can do. I believe that these developments should make us re-think the nature of creativity itself, and through identifying parallels and differences between generative models and the human brain we can establish a framework to talk about creativity, and its relationship to intelligence, that should hold up against future revelations in ML and neuroscience. // Bio Nick Hasty is a technologist & entrepreneur with a background in the creative arts. He was the founding engineer for GIPHY, where he’s worked for the last 10 years and now leads ML/AI product initiatives. He also servers as a consultant helping early-stage startups scale their product and engineering teams. Before GIPHY, he worked with arts+cultural organizations such as Rhizome.org and the Alan Lomax archives. He got his graduate degree from NYU’s ITP program. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: http://jnhasty.com/ Previous talks: https://engineering.giphy.com/giphy2vec-natural-language-processing-giphy/ https://changelog.com/practicalai/38 ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Nick on LinkedIn: https://www.linkedin.com/in/nickhasty/ Timestamps: [00:00] Nick's preferred coffee [00:15] Takeaways [06:58] Nick's background in ML [12:15] Nick's GIPHY journey [17:39] Nick's Success Factors [20:50] The trajectory of AI [28:09] Identifying as a product engineer [32:42] Evaluate LLMs vs. Traditional Models [35:03] AI Product: Intuition vs Data [38:53] Giphy AI Product Development [45:25] Startups and Venture Assistance [52:30] AI Funding Landscape Shift [54:00] Wrap up

MLOps podcast #197 with Niels Bantilan, Chief Machine Learning Engineer at Union, The Role of Infrastructure in ML Leveraging Open Source brought to us by Union. // Abstract When we start out building and deploying models in a new organization, life is simple: all I need to do is grab some data, iterate on a model that fits the data well and performs reasonably well on some held-out test set. Then, if you’re fortunate enough to get to the point where you want to deploy it, it’s fairly straightforward to wrap it in an app framework and host it on a cloud server. However, once you get past this stage, you’re likely to find yourself needing: More scalable data processing framework Experiment tracking for models Heavier duty CPU/GPU hardware Versioning tools to link models, data, code, and resource requirements Monitoring tools for tracking data and model quality There’s a rich ecosystem of open-source tools that solves each of these problems and more: but how do you unify all of them together into a single view? This is where orchestration tools like Flyte can help. Flyte not only allows you to compose data and ML pipelines, but it also serves as “infrastructure as code” so that you can leverage the open-source ecosystem and unify purpose-built tools for different parts of the ML lifecycle on a single platform. ML systems are not just models: they are the models, data, and infrastructure combined. // Bio Niels is the Chief Machine Learning Engineer at Union.ai, and core maintainer of Flyte, an open-source workflow orchestration tool, author of UnionML, an MLOps framework for machine learning microservices, and creator of Pandera, a statistical typing and data testing tool for scientific data containers. His mission is to help data science and machine learning practitioners be more productive. He has a Masters in Public Health with a specialization in sociomedical science and public health informatics, and prior to that a background in developmental biology and immunology. His research interests include reinforcement learning, AutoML, creative machine learning, and fairness, accountability, and transparency in automated systems. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://github.com/cosmicBboy, https://union.ai/Flyte: https://flyte.org/ MLOps vs ML Orchestration // Ketan Umare // MLOps Podcast #183 - https://youtu.be/k2QRNJXyzFg ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Niels on LinkedIn: https://www.linkedin.com/in/nbantilan/ Timestamps: [00:00] Niels' preferred coffee [00:17] Takeaways [03:45] Shout out to our Premium Brand Partner, Union! [04:30] Pandera [08:12] Creating a company [14:22] Injecting ML for Data [17:30] ML for Infrastructure Optimization [22:17] AI Implementation Challenges [24:25] Generative DevOps movement [28:27] Pushing Limits: Code Responsibility [29:46] Orchestration in OpenAI's Dev Day [34:27] MLOps Stack: Layers & Challenges [42:45] Mature Companies Embrace Kubernetes [45:29] Horizon Challenges [47:24] Flexible Integration for Resources [49:10] MLOps Reproducibility Challenges [53:14] MLOps Maturity Spectrum [57:48] First-Class Citizens in Design [1:00:16] Delegating for Efficient Collaboration [1:04:55] Wrap up

Laurel Orr is a Principal Engineer at Numbers Station, a startup that applies Foundation Model technology to the enterprise data stack. Venky Orr is SVP of Product & Engineering.

MLOps podcast #196 with Numbers Station's Venky Ganti SVP, Product & Engineering and Principal Engineer, Laurel Orr, LLMs in Focus: From One-Size Fits All to Verticalized Solutions. // Abstract Dive into the realm of large language models (LLMs) as we explore the merits and limitations of 'one-size fits all' LLMs, and their role in data analytics. Through customer stories, we showcase real-world applications and contrast general LLMs with verticalized, enterprise-centric models. We address the significance of ownership structures, with a focus on open-source vs proprietary impacts on transparency and trustworthiness. Delving into the NSQL foundation models, we emphasize the importance of diverse, quality training data, especially with enterprise challenges. Lastly, we speculate on the future of LLMs, highlighting hosting solutions and the evolution towards specialized challenges. // Bio Laurel Orr Laurel Orr is a Principal Engineer at Numbers Station, a startup that applies Foundation Model technology to the enterprise data stack. Her research interests include how to use FMs to solve classically hard data-wrangling tasks and how to put FM technology into deployment. Before Numbers Station, Laurel was a postdoc at Stanford advised by Chris Re as part of the Hazy Research Labs working in the intersection of AI and data management. She graduated with a PhD in database systems from the University of Washington. Venky Orr Venky brings over two decades of experience in software engineering and technical leadership to Numbers Station as SVP of Product & Engineering. Most recently, he served as General Manager leading several initiatives on query understanding and commerce in the ads product area at Google. Before that, he was CEO and co-founder of Mesh Dynamics, the API test automation company, which was acquired by Google in 2021. Prior to Mesh Dynamics, Venky was CTO and co-founder of Alation, the enterprise data catalog company, where he led technology and helped create the new data catalog product category. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.numbersstation.ai/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Laurel on LinkedIn: https://www.linkedin.com/in/laurel-orr/ Connect with Venky on LinkedIn: https://www.linkedin.com/in/venky-ganti-2679a2/ Timestamps: [00:00] Venky's and Laurel's preferred coffee [00:36] Takeaways [03:15] Please like, share, and subscribe to our MLOps channels! [04:38] Venky's background [07:47] Laurel's at background [09:38] Data wrangling [13:45] Sequel query [19:25] One size-fits-all LLMs vs Verticalized and Specific LLMs [23:42] Model Choice Trade-offs [30:18] NSQL Foundational Models [37:26] LLM Trends in 12 Months [40:09] Data recipes being democratized [45:16] Claude and 100,000 Context [48:02] Exploring Varieties of LLMs [50:02] AI Gateway [51:07] Text-to-SQL Model Evaluation [54:00] Wrap up

MLOps Coffee Sessions Special episode with Weights & Biases, Model Management in a Regulated Environment, fueled by our Premium Brand Partner, Weights & Biases. // Abstract Step into the fascinating world of Language Model Management (LLMs) in a Regulated Environment! Join us for an enlightening chat where we'll explore the intricacies of managing models within highly regulated settings, focusing on compliance and effective strategies. This is your opportunity to be part of a dynamic conversation that delves into the challenges and best practices of Model Management in Regulated Environments. Secure your spot today and stay tuned for an enriching dialogue on navigating the complexities of navigating the regulated terrain. Don't miss out on the chance to broaden your understanding and connect with peers in the field! // Bio Darek Kłeczek Darek Kłeczek is a Machine Learning Engineer at Weights & Biases, where he leads the W&B education program. Previously, he applied machine learning across supply chain, manufacturing, legal, and commercial use cases. He also worked on operationalizing machine learning at P&G. Darek contributed the first Polish versions of BERT and GPT language models and is a Kaggle Competitions Grandmaster. Mark Huang Mark is a co-founder and Chief Architect at Gradient, a platform that helps companies build custom AI applications by making it extremely easy to fine-tune foundational models and deploy them into production. Previously, he was a tech lead in machine learning teams at Splunk and Box, developing and deploying production systems for streaming analytics, personalization, and forecasting. Prior to his career in software development, he was an algorithmic trader at quantitative hedge funds where he also harnessed large-scale data to generate trading signals for billion-dollar asset portfolios. Oliver Chipperfield Oliver Chipperfield is a Senior Data Scientist and Team Lead at M-KOPA, where he utilizes his expertise in machine learning and data-driven innovation. At M-KOPA since October 2021, Oliver leads a diverse tech team, making improvements in credit loss forecasting and fraud detection. His career spans multiple industries, where he has applied his extensive knowledge in Python, Spark, R, SQL, and Excel. He also specialized in the building and design of production ML systems, experimentation, and Bayesian statistics. Michelle Marie Conway As an Irish woman who relocated to London after completing her university studies in Dublin, Michelle spent the past 12 years carving out a career in the data and tech industry. With a keen eye for detail and a passion for innovation, She has consistently leveraged my expertise to drive growth and deliver results for the companies she worked for. As a dynamic and driven professional, Michelle is always looking for new challenges and opportunities to learn and grow, and she's excited to see what the future holds in this exciting and ever-evolving industry. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Fine-Tuning LLMs: Best Practices and When to Go Small // Mark Kim-Huang // MLOps Meetup #124 - https://youtu.be/1WSUfWojoe0 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Darek on LinkedIn: https://www.linkedin.com/in/kleczek/ Connect with Mark on LinkedIn: https://www.linkedin.com/in/markhng525/ Connect with Oliver on LinkedIn: https://www.linkedin.com/in/oliver-chipperfield/ Connect with Michelle on LinkedIn: https://www.linkedin.com/in/michelle-conway-40337432

MLOps podcast #195 with Varun Mohan, CEO of Codeium, Building the Future of AI in Software Development brought to us by QuantumBlack. // Abstract This brief overview traces the evolution of Exafunction and Codeium, highlighting the strategic transition. It explores the inception of Codeium's key features, offering insights into the thoughtful design process. This emphasizes the company's forward-looking approach to preparing for a rapidly advancing technological landscape. Additionally, it touches upon developing essential MLOps systems, showcasing the commitment to maintaining rigor and efficiency in the face of evolving challenges. // Bio Varun Mohan developed a knack for programming in high school where he actively participated in various competitions. This passion for programming was shared with his now co-founder, with whom he frequently competed. Their common interest in programming and competition led them to attend MIT together, where they undertook more programming challenges. After college, they ventured into the Bay Area where they continued to compete and further cultivate their programming abilities. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Websites: codeium.com, https://exafunction.com/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Varun on Twitter: https://www.linkedin.com/in/varunkmohan/ Timestamps: [00:00] Varun's preferred coffee [00:15] Takeaways [02:50] Please like, share, and subscribe to our MLOps channels! [03:05] QuantumBlack ad by Nayur Khan [05:51] Varun's background in tech [10:55] Language Models Advancement [14:17] GPU scarce world [18:23] Vision and Pain Points [19:18] Fine-tuning Challenges in NLP [21:04] ML and AI Caution [21:49] MLOps: App vs Infra [23:53] Data Engineering Abstraction Evolution [26:12] Codeium and Scaling Discussion [31:59] API, Cloud, Computation [34:20] Codeium scaling [35:11] Reserved GPUs, companies self-hosting products [38:00] Open-source code Codeium training [40:03] Protecting IP Licenses [41:32] ML Challenges: Data, Bias, Security [44:37] Evaluating code [48:29] Getting values from Codeium [49:49] Exafunction ML AI Production [52:17] AWS Creation [53:58] Feature flags and MA AI lifecycle [56:34] Coding problem [58:40] New software architectures [1:03:28] Wrap up

// Abstract Explore the transformative role of AI in EdTech, discussing its potential to enhance learning experiences and personalize education. The panelists share insights on AI use cases, challenges in AI integration, and strategies for building a differentiated business model in the evolving AI landscape. The discussion looks ahead at how the latest wave of GenAI is set to shape the future of education. Join us to understand the exciting prospects and challenges of AI in EdTech. Moderator: Paul van der Boor // Bio Klinton Bicknell Klinton Bicknell is the Head of AI  @duolingo . He works at the intersection of artificial intelligence and cognitive science. His research has been published in venues including ACL, PNAS, NAACL, Psychological Science, EDM, CogSci, and Cognition, and covered in the Financial Times, BBC, and Forbes. Prior to Duolingo, he was an assistant professor at Northwestern University. Bill Salak Bill Salak has more than 20 years of experience overseeing large-scale development projects and more than 24 years of experience in web application architecture and development. Bill founded and served as CTO of multiple Internet and web development companies, leading technology projects for companies including Age of Learning, AOL, Educational Testing Systems, Film LA, Hasbro, HBO, Highlights for Children, NBC-Universal, and the U.S. Army. Bill currently serves as the CTO of  @Brainly-app , the leading learning platform worldwide with the most extensive Knowledge Base for all school subjects and grades. Yeva Hyusyan Yeva Hyusyan is the Co-Founder and CEO of  @Sololearn , the most engaging platform for learning how to code. Prior to co-founding SoloLearn, Yeva established a startup accelerator for mobile games, consumer apps, and ag-tech solutions. In a previous role, she implemented programs for the World Bank and the US Government in business and education. Later, she served as a General Manager at Microsoft, where she led sales, developer ecosystem development, and strategic partnerships. Yeva holds an MBA in Corporate Strategy from Maastricht School of Management in the Netherlands, an MS in International Economics from Yerevan State University in Armenia, and completed the Executive Program at Stanford University's Graduate School of Business. // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

Join our conference: https://home.mlops.community/public/events/llms-in-production-part-iii-2023-10-03

MLOps Coffee Sessions Special episode with Tecton, Get your ML Application Into Production, sponsored by Tecton. // Abstract Getting an ML application into production is more difficult than most teams expect—but with the right preparation, it can be done efficiently! Join us for this exclusive roundtable, where 4 machine learning experts from Tecton will discuss some of the most common challenges and best practices to avoid them. With over 35 years of combined experience in MLOps at companies like AWS, Google, Lyft, and Uber, and 15 years of experience at Tecton spent helping customers like FanDuel, Plaid, and HelloFresh getting ML models into production, the presenters will share how factors like organizational structure, use cases, tech stack, and more, can create different types of bottlenecks. They’ll also share best practices and lessons learned throughout their careers on how to overcome these challenges. // Bio Kevin Stumpf Kevin co-founded Tecton where he leads a world-class engineering team that is building a next-generation feature store for operational Machine Learning. Kevin and his co-founders built deep expertise in operational ML platforms while at Uber, where they created the Michelangelo platform that enabled Uber to scale from 0 to 1000's of ML-driven applications in just a few years. Prior to Uber, Kevin founded Dispatcher, with the vision to build the Uber for long-haul trucking. Kevin holds an MBA from Stanford University and a Bachelor's Degree in Computer and Management Sciences from the University of Hagen. Outside of work, Kevin is a passionate long-distance endurance athlete. Derek Salama Derek is currently a Senior Product Manager at Tecton, where he is responsible for security, collaboration experience, and Feature Platform infrastructure. Prior to Tecton, Derek worked at Google and Lyft across both ML infrastructure and ML applications. Eddie Esquivel Eddie Esquivel is a Solutions Architect at Tecton, where he helps customers implement feature stores as part of their stack for operational ML. Prior to Tecton, Eddie was a Solutions Architect at AWS. He holds a Bachelor’s Degree in Computer Science & Engineering from the University of California, Los Angeles. Isaac Cameron Isaac Cameron is a Consulting Architect at Tecton. Prior to Tecton, he was a Principal Solutions Architect at Slalom Build, focusing on data and machine learning, where he built his own feature platform for a large U.S. airline and has enabled many organizations to build intelligent products leveraging operational ML. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Kevin on LinkedIn: https://www.linkedin.com/in/kevinstumpf/ Connect with Derek on LinkedIn: https://www.linkedin.com/in/dereksalama/ Connect with Eddie on LinkedIn: https://www.linkedin.com/in/eddie-esquivel-2016/ Connect with Isaac on LinkedIn: https://www.linkedin.com/in/isaaccameron/ Timestamps: [00:00] Introduction to Kevin Stumpf, Derek Salama, Eddie Esquivel, and Isaac Cameron [02:48] Challenges of traditional classical ML into production [10:21] Infrastructure cost [16:50] Bridging Business and Tech [19:23] ML Infrastructure Essentials [29:38] Integrated Batch and Stream [35:12] Scaling AI from Zero [36:23] Stacks red flags [45:53] Tecton: Features Quality Monitoring [49:06] Building Recommender System Tools [53:19] Quantify business value in ML [54:40] Wrap up

MLOps podcast #194 with Omar Khattab, PhD Candidate at Stanford, DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. // Abstract The ML community is rapidly exploring techniques for prompting language models (LMs) and for stacking them into pipelines that solve complex tasks. Unfortunately, existing LM pipelines are typically implemented using hard-coded "prompt templates", i.e. lengthy strings discovered via trial and error. Toward a more systematic approach for developing and optimizing LM pipelines, we introduce DSPy, a programming model that abstracts LM pipelines as text transformation graphs, i.e. imperative computational graphs where LMs are invoked through declarative modules. DSPy modules are parameterized, meaning they can learn (by creating and collecting demonstrations) how to apply compositions of prompting, finetuning, augmentation, and reasoning techniques. We design a compiler that will optimize any DSPy pipeline to maximize a given metric. We conduct two case studies, showing that succinct DSPy programs can express and optimize sophisticated LM pipelines that reason about math word problems, tackle multi-hop retrieval, answer complex questions, and control agent loops. Within minutes of compiling, a few lines of DSPy allow GPT-3.5 and llama2-13b-chat to self-bootstrap pipelines that outperform standard few-shot prompting and pipelines with expert-created demonstrations. On top of that, DSPy programs compiled to open and relatively small LMs like 770M-parameter T5 and llama2-13b-chat are competitive with approaches that rely on expert-written prompt chains for proprietary GPT-3.5. DSPy is available as open source at https://github.com/stanfordnlp/dspy // Bio Omar Khattab is a PhD candidate at Stanford and an Apple PhD Scholar in AI/ML. He builds retrieval models as well as retrieval-based NLP systems, which can leverage large text collections to craft knowledgeable responses efficiently and transparently. Omar is the author of the ColBERT retrieval model, which has been central to the development of the field of neural retrieval, and author of several of its derivate NLP systems like ColBERT-QA and Baleen. His recent work includes the DSPy framework for solving advanced tasks with language models (LMs) and retrieval models (RMs). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://omarkhattab.com/ DSPy: https://github.com/stanfordnlp/dspy ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Omar on Twitter: https://twitter.com/lateinteraction Timestamps: [00:00] Omar's preferred coffee [00:26] Takeaways [06:40] Weight & Biases Ad [09:00] Omar's tech background [13:35] Evolution of RAG [16:33] Complex retrievals [21:32] Vector Encoding for Databases [23:50] BERT vs New Models [28:00] Resilient Pipelines: Design Principles [33:37] MLOps Workflow Challenges [36:15] Guiding LLMs for Tasks [37:40] Large Language Models: Usage and Costs [41:32] DSPy Breakdown [51:05] AI Compliance Roundtable [55:40] Fine-Tuning Frustrations and Solutions [57:27] Fine-Tuning Challenges in ML [1:00:55] Versatile GPT-3 in Agents [1:03:53] AI Focus: DSP and Retrieval [1:04:55] Commercialization plans [1:05:27] Wrap up

// Abstract Martian is focused on building a model router to dynamically route every prompt to the best LLM for the highest performance and lowest cost. Corti, the Al Co-Pilot for health care uses Al to improve patient care, demonstrating the potential of Al in healthcare and medical decision-making. They recently raised $60M, with Prosus being one of the lead investors. Transforms is pioneering in synthetic entertainment, showing how Al can transform the way we create and consume media. Moderator: Paul van der Boor // Speakers Sandeep Bakshi Head of Investments, Europe  @prosusgroup3707  Shriyash Upadhyay Founder @Martian Lars Maaløe Co-Founder & CTO at Corti | Adj. Assoc. Professor of Machine Learning @ Corti Pietro Gagliano President & Founder @Transitional Forms // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

MLOps podcast #193 with Pierre Salvy, Head of Engineering at Cambrium, LLM in Material Production co-hosted by Stephen Batifol. // Abstract Delve into the world of proteins, genetic engineering, and the intersection of AI and biotech. Pierre explains how his company is using advanced models to design proteins with specific properties, even creating a vegan collagen for cosmetics. By harnessing the potential of AI, they aim to revolutionize sustainability, uncovering a future of lab-grown meats, molecular cheese, and less harmful plastics, confronting regulatory barriers and decoding the syntax and grammar of proteins. // Bio Head of Engineering at Cambrium, a biotech company utilising genAI to design sustainable protein biomaterials for the future. Pierre spent the last decade researching ways to make computers calculate better biological systems. This is a critical step to engineering more sustainable ways to make the products we use every day, which is their mission at Cambrium. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: cambrium.bio --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/ Connect with Pierre on LinkedIn: https://www.linkedin.com/in/psalvy/ Timestamps: [00:00] Pierre's preferred coffee [00:10] Takeaways [05:10] Please like, share, and subscribe to our MLOps channels! [05:25] Weights and Biases ad [07:52] Ski story [09:54] Pierre's career trajectory [13:35] From employee #2 to hiring a team [14:42] From employee #2 to head of engineering [15:50] Uncomfortable things to say essential for growth and effectiveness [18:27] From biotech to engineering [21:10] LLMs at Cambrium [24:26] Slackbot [25:43] Quick and Easy Solutions [26:47] Products created at Cambrium [31:56] Impact of EU Regulation on Cambrium [35:39] 2nd Biotech Winter [36:35] Cost of error vs service not working [38:00] Protein Synthesis and Mutations [40:03] Large-Scale System Engineering Challenges [43:28] Expensive Factors in Experiments [44:39] LLMs vs Protein Models [47:03] Protein Design with LLMs [49:43] Eco-Friendly Product Vision [53:28] Space glue [54:00] Wrap up

// Abstract A product-minded engineering perspective on UX/design patterns, product evaluation, and building with AI. // Bio Charles Frye Charles teaches people how to build ML applications. After doing research in psychopharmacology and neurobiology, he pivoted to artificial neural networks and completed a PhD at the University of California, Berkeley in 2020. He then worked as an educator at Weights & Biases before joining @Full Stack Deep Learning, an online community and MOOC for building with ML. Sahar Mor Sahar is a Product Lead at  @stripe  with 15y of experience in product and engineering roles. At Stripe, he leads the adoption of LLMs and the Enhanced Issuer Network - a set of data partnerships with top banks to reduce payment fraud. Prior to Stripe he founded a document intelligence API company, was a founding PM in a couple of AI startups, including an accounting automation startup (Zeitgold, acq'd by Deel), and served in the elite intelligence unit 8200 in engineering roles. Sahar authors a weekly AI newsletter (AI Tidbits) and maintains a few open-source AI-related libraries (https://github.com/saharmor). Sarah Guo Sarah Guo is the Founder and Managing Partner at @Conviction, a venture capital firm founded in 2022 to invest in intelligent software, or "Software 3.0." Prior, she spent a decade as a General Partner at Greylock Partners. She has been an early investor or advisor to 40+ companies in software, fintech, security, infrastructure, fundamental research, and AI-native applications. Sarah is from Wisconsin, has four degrees from the University of Pennsylvania, and lives in the Bay Area with her husband and two daughters. She co-hosts the AI podcast "No Priors" with Elad Gil. Shyamala Prayaga Shyamala is a seasoned conversational AI expert. Having led initiatives across connected home, automotive, wearables - just to name a few, she's put her work on research into usability, accessibility, speech recognition, multimodal voice user interfaces, and has even been published internationally across publications like Forbes. Outside of her research, she's spent the last 18 years designing mobile, web, desktop, and smart TV interfaces and has most recently joined  @NVIDIA  to work on deep learning product suites. Willem Pienaar Willem is the creator of @Feast, the open-source feature store and a builder in the generative AI space. Previously Willem was an engineering manager at Tecton where he led teams in both their open source and enterprise initiatives. Before that Willem built the core ML systems and created the ML platform team at Gojek, the Indonesian decacorn. // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

MLOps podcast #192 with Chris Van Pelt, CISO and co-founder of Weights & Biases, Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines sponsored by  @WeightsBiases . // Abstract Chris, provides insights into his machine learning (ML) journey, emphasizing the significance of ML evaluation processes and the evolving landscape of MLOps. The conversation covers effective evaluation metrics, demo-driven development nuances, and the complexities of ML Ops pipelines. Chris reflects on his experience with Crowdflower, detailing its transition to Weights and Biases and stressing the early integration of security measures. The discussion extends to the transformative impact of ML on the tech industry, challenges in detecting subtle bugs, and the potential of open-source models and multimodal capabilities. // Bio Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://wandb.ai/site ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/ Timestamps: [00:00] Chris' preferred coffee [00:33] Takeaways[03:50] Huge shout out to Weights & Biases for sponsoring this episode! [04:15] Please like, share, and subscribe to our MLOps channels! [04:25] CrowdFlower [07:02] Difference of CrowdFlower and Trajectory [09:13] Transition from CrowdFlower to Weights & Biases [13:05] Excel spreadsheets being passed around via email [15:45] Evolution of Weights & Biases [19:24] CISO role [22:23] Advise for easy wins [25:32] Transition into LLMs [27:36] Prompt injection risks on data [29:42] LLMs for New Personas [34:42] Iterative Value Evaluation Process [36:36] Iterating on New Release [39:31] Evaluation survey [43:21] Landscape of LLMs and its evolution [45:40] Conan O'Brien [46:48] Wrap up

MLOps podcast #191 with Gregory Kamradt, Founder of  @DataIndependent, Building Defensible AI Apps sponsored by  @MilvusVectorDatabase . // Abstract Demetrios engages in a captivating conversation with Gregory Kamradt, an AI visionary deeply immersed in technology and product development. The discussion spans various challenges businesses encounter in implementing AI, the transformative potential of AI in revolutionizing business processes, and the growth and possibilities associated with OpenAI. Gregory shares insights into his latest project, a smart companion app designed to analyze and summarize startup pitches. The episode unfolds as a rich source of knowledge, exploring diverse topics such as AI experimentation, the concept of an AI gateway, the future of finely tuned models for niche applications, and insights into the intricate landscape of AI within big tech, including Google's strategic direction and OpenAI's copyright protection measures. // Bio Greg has mentored thousands of developers and founders, empowering them to build AI-centric applications. By crafting tutorial-based content, Greg aims to guide everyone from seasoned builders to ambitious indie hackers. Greg partners with companies during their product launches, feature enhancements, and funding rounds. His objective is to cultivate not just awareness, but also a practical understanding of how to optimally utilize a company's tools. He previously led Growth @ Salesforce for Sales & Service Clouds in addition to being early on at Digits, a FinTech Series-C company. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://gregkamradt.com/ Greg Kamradt (Data Indy): https://www.youtube.com/@DataIndependent Milvus Vector Database: https://zilliz.com/what-is-milvus ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Greg on LinkedIn: https://www.linkedin.com/in/gregkamradt/ Timestamps: [00:00] Greg's preferred coffee [00:12] Takeaways [02:56] Quick word from our sponsor [04:22] DevDay [06:19] YouTube's unique perspective on the technological revolution [09:34] GPT assistance [13:36] AI Streamlining Fax Orders [18:13] AI Marketplace Dynamics: GPT vs. Specialized [22:04] Data Tooling Platform Challenges [27:17] The Shield against copyright [29:27] Llama Index vs OpenAI [31:56] DS Pie and Compiler Tangent [34:31] Orchestration Layer is dead! [36:49] Personalized AI Models: Understanding Integration [38:00] AI Defensibility [43:00] Green Field AI Opportunities [46:57] LLMs for live event pitch [53:38] Exciting content creation process [58:03] New context window benchmark [1:02:23] AI Gateway [1:04:35] Wrap up

MLOps podcast #190 with Ads Dawson, Senior Security Engineer at Cohere, Guarding LLM and NLP APIs: A Trailblazing Odyssey for Enhanced Security. // Abstract Ads Dawson, a seasoned security engineer at Cohere, explores the challenges and solutions in securing large language models (LLMs) and natural language programming APIs. Drawing on his extensive experience, Ads discusses approaches to threat modeling LLM applications, preventing data breaches, defending against attacks, and bolstering the security of these critical technologies. The presentation also delves into the success of the "OWASP Top 10 for Large Language Model Applications" project, co-founded by Ads, which identifies key vulnerabilities in the industry. Notably, Ads owns three of the top 10 vulnerabilities, including Training Data Poisoning, Sensitive Information Disclosure, and Model Theft. This OWASP Top 10 serves as a foundational resource for stakeholders in AI, offering guidance on using, developing, and securing LLM applications. Additionally, the session covers insider news from the AI Village's 'Hack the Future' | LLM Red Teaming event at Defcon31, providing insights into the inaugural Generative AI Red Teaming showdown and its significance in addressing security and privacy concerns amid the widespread adoption of AI. // Bio A mainly self-taught, driven, and motivated proficient application, network infrastructure & cyber security professional holding over eleven years experience from start-up to large-size enterprises leading the incident response process and specializing in extensive LLM/AI Security, Web Application Security and DevSecOps protecting REST API endpoints, large-scale microservice architectures in hybrid cloud environments, application source code as well as EDR, threat hunting, reverse engineering, and forensics. Ads have a passion for all things blue and red teams, be that offensive & API security, automation of detection & remediation (SOAR), or deep packet inspection for example. Ads is also a networking veteran and love a good PCAP to delve into. One of my favorite things at Defcon is hunting for PWNs at the "Wall of Sheep" village and inspecting malicious payloads and binaries. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://github.com/GangGreenTemperTatum OWASP Top 10 for Large Language Model Applications Core Team Member and Founder - https://owasp.org/www-project-top-10-for-large-language-model-applications/CoreTeam Fork for OWASP Top 10 for Large Language Model Applications - https://github.com/GangGreenTemperTatum/www-project-top-10-for-large-language-model-applications Security project: llmtop10.com --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ads on LinkedIn: https://www.linkedin.com/in/adamdawson0/ Timestamps: [00:00] Ads' preferred coffee [00:46] Takeaways [02:52] Please like, share, and subscribe to our MLOps channels! [03:11] Security and vulnerabilities [05:24] Work at Cohere and OWASP [08:11] Previous work vs LLMs Companies [09:46] LLM vulnerabilities [10:38] Good qualities to combat prompt injection problems [13:26] Data lineage [16:03] Red teaming [19:39] Freakiest LLM vulnerabilities [22:17] Severe Autonomy Concerns [25:13] Hallucinations [27:59] Prompt injection [29:15] Vector attacks to be recognized [32:02] LLMs being customed [33:18] Security changes due to maturity [38:17] OWASP Top 10 for Large Language Model Applications [44:31] Gandalf game [46:06] Prompt injection attack [49:46] Overlapping security [53:26] Data poisoning [56:57] Toxic data for LLMs [58:50] Wrap up

MLOps podcast #189 with Rohit Agarwal, CEO of Portkey.ai, Designing for Forward Compatibility in Gen AI. // Abstract For two whole years of working with a large LLM deployment, I always felt uncomfortable. How is my system performing? Are my users liking the outputs? Who needs help? Probabilistic systems can make this really hard to understand. In this talk, we'll discuss practical & implementable items to secure your LLM system and gain confidence while deploying to production. // Bio Rohit is the Co-founder and CEO of portkey.ai which is an FMOps stack for monitoring, model management, compliance, and more. Previously, he headed Product & AI at Pepper Content which has served ~900M generations on LLMs in production. Having seen large LLM deployments in production, he's always happy to help companies build their infra stacks on FM APIs or Open-source models. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://portkey.ai ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Rohit on LinkedIn: https://www.linkedin.com/in/1rohitagarwal/ Timestamps: [00:00] Rohit's preferred coffee [00:15] Takeaways [03:22] Please like, share, and subscribe to our MLOps channels! [05:16] Rohit's current work [06:37] The Portkey landscape [09:13] Compute unit is no longer a Cloud resource, it's a Foundational Model [11:09] Hang-ups at high-scale models and how to combat them [15:22] Complexity of the Apps evolving [19:54] Rohit's working relationships with the agents [22:52] Fine-tuning reliability [24:38] Small language models can outperform larger ones [26:38] Market map at Portkey [34:37] AI Gateway [37:59] Worker Bee and Queen Bee [39:27] Security and Compliance [43:11] Idea of Data Mesh [45:57] Forward compatibility [49:59] Decoupling AI Gateway from the code [56:05] Hardest design decisions to make since creating Portkey [58:52] Wrap up

MLOps podcast #188 with Anand Das, Co-founder and CTO of Bito, Impact of LLMs on the Tech Stack and Product Development. // Abstract Anand and his team have developed a fascinating Chrome extension called "explain code" that has garnered significant attention in the tech community. They have expanded their extension to other platforms like Visual Studio code and Chat Brains, creating a personal assistant for code generation, explanation, and test case writing. // Bio Anand Das is the co-founder and CTO of Bito. Previously, he served as the CTO at Eyeota, which was acquired by Dun & Bradstreet for $165M in 2021. Anand also co-founded and served as the CTO of PubMatic in 2006, a company that went public on NASDAQ in 2020 (NASDAQ: PUBM). Anand has also held various engineering roles at Panta Systems, a high-performance computing startup led by the CTO of Veritas, as well as at Veritas and Symantec, where he worked on a variety of storage and backup products. Anand holds seven patents in systems software, storage software, advertising, and application software. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://bito.ai/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Anand on LinkedIn: https://www.linkedin.com/in/ananddas/ Timestamps: [00:00] Anand's preferred coffee [00:15] Takeaways [02:49] Please like, share, and subscribe to our MLOps channels! [03:08] Anand's tech background [10:06] Fun at Optimization Level [12:59] Trying all APIs [17:55] Models evaluation decision tree [22:51] Weights and Biases Ad [25:04] AI Stack that understands the code [28:27] Tools for the Guard Rails [33:23] Seeking solutions before presenting to LLM [38:46] Prompt-Driven Development Insights [40:16] Prompting best practices [42:51] Unneeded complexities [45:45] Cost-benefit analysis of buying GPUs [49:13] ML Build vs Buy [51:26] Best practices for debugging code assistant [54:58] Wrap up

MLOps podcast #187 with Faizaan Charania, Product Manager, AI at LinkedIn, Building Effective Products with GenAI. // Abstract Faizaan outlines his AI product development approach, starting broadly and refining details with tech leads, emphasizing the value of a simplified MVP. He also explores integrating generative AI, highlighting its role in enhancing user experiences through LLMs. In this discussion, Faizaan shares wisdom on feedback integration, user trust, and the collaboration challenges between product managers and AI teams. Let's delve into evaluating AI-driven experiences and the complexities that arise in this dynamic landscape! // Bio Faizaan is an AI Product lead at LinkedIn working on Personalization and Generative AI use cases for Creators and Conversations on LinkedIn. He's been in the field of machine learning for 8+ years now he started as a research assistant eventually transitioning to being a Product Manager during his time at Yahoo. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Faizaan on LinkedIn: https://www.linkedin.com/in/faizaan-charania/ Timestamps: [00:00] Faizaan's preferred beverage [00:22] Takeaways [02:54] The Bollywood actor [05:23] Faizaan's in tech [07:45] Technical pieces to learn about before working at LinkedIn [09:23] Tech Team Data Strategy [12:01] Gradual vs. Advanced ML Implementation [13:36] Shipping on time [14:11] Thoughts on building products with AI [18:20] Push and pull mechanism [21:47] Costs and Choices with AI Models [25:06] AI ROI Evaluation [27:02] Thoughts on open source [28:17] Building Generative AI focus [31:50] Prompts and Anomalies [34:57] Where to have a human in the loop [35:45] Problem-driven AI Tool [37:56] Creator of AI-generated post on LinkedIn [39:50] Product Impact on AI Democratization [41:15] Distinct signals to measure ROI [44:38] PMs learning AI while ML teams learn product [47:22] Gotchas seen when adding a new AI feature [50:00] Evaluation Challenges in Responses [51:55] Who's more confident? [52:55] Wrap up

MLOps podcast #186 with Mike Del Balso, CEO & Co-founder of Tecton and Josh Wills, Angel Investor, The Future of Feature Stores and Platforms. // Abstract Mike and Josh discuss creating templates and working at a detailed level, exploring Tecton's potential for sharing fraud and third-party features. They focus on technical aspects like data handling and optimizing models, emphasizing the significance of quality data for AI systems and the necessity for cohesive feature infrastructure in reaching production stages. // Bio Mike Del Balso Mike is the co-founder of Tecton, where he is focused on building next-generation data infrastructure for Operational ML. Before Tecton, Mike was the PM lead for the Uber Michelangelo ML platform. He was also a product manager at Google where he managed the core ML systems that power Google’s Search Ads business. Josh Wills Josh Wills is an angel investor specializing in data and machine learning infrastructure. He was formerly the head of data engineering at Slack, the director of data science at Cloudera, and a software engineer at Google. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mike on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/Connect with Josh on LinkedIn: https://www.linkedin.com/in/josh-wills-13882b/ Timestamps: [00:00] Introduction to Mike [01:45] Takeaways [03:32] Features of the new paradigm of ML and LLMs [06:00] D. Sculley's papers [13:05] The birth of Feature Store [17:06] Data Pipeline Challenges Addressed [20:00] Operationalizing [26:50] Feature Store Challenges [30:26] Z access [36:23] Addressing Technical Debt Challenges [37:27] Real-Time vs. Batch Processing [47:10] Feature Store Evolution: Apache Iceberg [49:59] Feature Platform: Dedicated Query Engine [54:04] The bottleneck [56:00] LLMs, Feature Stores Overview [1:00:20] Vector databases [1:06:15] Workflow Templating Efficiency [1:08:35] Gamification suggestion for Tecton [1:10:25] Wrap up

MLOps podcast #185 with Luigi Patruno, VP of Data Science at 2U, Inc, Lessons on Data Science Leadership. // AbstractPicture this: you've got data products to manage, and you're in charge of a team. It's not all sunshine and rainbows, right? Luigi dives into the nitty-gritty of the challenges - from juggling data projects to wrangling the team dynamics. It's a real adventure, let me tell you! // Bio Luigi Patruno is a results-driven data science leader passionate about identifying value-add business opportunities and converting these into analytical solutions that deliver measurable business outcomes. As a leader he focuses on defining strategic vision and, through motivation and discipline, driving teams of highly quantitative data scientists, machine learning engineers, and product managers to achieve extraordinary results. He is currently the VP of Data Science at 2U, where he leads the data science department focused on optimizing business operations through advanced analytics, experimentation, and machine learning. He enjoys teaching others how to leverage data science to improve their businesses through public speaking, teaching courses, and writing online at MLinProduction.com. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://mlinproduction.com/ YouTube channel: https://www.youtube.com/playlist?list=PLBLnN4jzkyqkjLIRpDNZcsG7TMMEk9Asa High Output Management book by Andrew Grove: https://www.amazon.nl/-/en/Andrew-S-Grove/dp/0679762884The One Minute Manager by Kenneth Blanchard Ph.D. and Spencer Johnson M.D.: https://www.amazon.com/Minute-Manager-Kenneth-Blanchard-Ph-D/dp/074350917X ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Luigi on LinkedIn: https://www.linkedin.com/in/luigipatruno91/ Timestamps: [00:00] Luigi's preferred coffee [00:30] Takeaways [03:04] Being practical [05:44] Data-Driven Decision-Making in Management [12:53] Recent Team Win [14:43] The perfect storm [20:22] Change Management and ROI [25:09] Change Management: Navigating Resistance [29:59] Clarifying North Star Communication [36:24] OKRs in Data Science [40:47] Success Likelihood in Business [45:08] Bus problem solution [49:25] Data Science-Platform Collaboration [53:19] Decentralized Platforms Explained [54:38] Data Platform Architecture Overview [57:14] Incentives for Team Motivation [1:09:45] The blind spots [1:12:22] Wrap up

MLOps podcast #184 with Richa Sachdev, Executive Director- Data Operations and Automation at JP Morgan Chase, Data Platforms in MLOps: Translating Business Goals into Product Decisions. // Abstract Richa, with her background in software engineering and experience in the financial sector, shares her insights on optimizing the end-user experience and the importance of understanding business goals and metrics. She discusses her journey in converting legacy applications, working with data platforms, and the challenges of integrating different databases. Richa also explores the role of automation in streamlining processes and improving customer interactions in the reward space. Join us as we unravel the fascinating world of MLOps and uncover the strategies and technologies that drive success in this ever-evolving field. // Bio A passionate and impact-driven leader whose expertise spans leading teams, architecting ML and data-intensive applications, and driving enterprise data strategy. Richa has worked for a Tier A Start-up developing feature platforms and in financial companies, leading ML Engineering teams to drive data-driven business decisions. Richa enjoys reading technical blogs focussed on system design and plays an active role in the MLOps Community. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.youtube.com/watch?v=i0To3DeHGuU
https://www.youtube.com/watch?v=tAOf2lVQUY4
https://www.youtube.com/watch?v=cXanVyaannQ
https://www.youtube.com/watch?v=2aWSsL24fv8 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Richa on LinkedIn: https://www.linkedin.com/in/richasachdev/ Timestamps: [00:00] Richa's preferred coffee [02:09] Takeaways [04:26] Richa's background to data [08:55] Prescriptive, Descriptive, and Predictive Data [11:50] Data Engineering Perspectives & Setup [17:34] Structured and Unstructured data [21:01] Richa's day-to-day at Chase [23:52] Figure out the business needs before the cool tech [26:46] Importance of business metrics [30:43] Optimizing end-user experience and trade-offs [36:06] Exhausting creativity in finding solutions [37:40] Consider faster implementation and increased ROI [40:20] Banks still using COBOL [41:17] Learning and growing as a versatile leader [42:04] Wrap up

MLOps podcast #183 with Ketan Umare, CEO of Union.AI, MLOps vs ML Orchestration co-hosted by Stephen Batifol. // Abstract Let's explore the relationship between Union and Flyte, emphasizing the significance of community-driven development and the challenge of balancing feature requests with security considerations. This conversation highlights the importance of real-time data and secure data handling in orchestrating machine learning models. The Flyte community's empathy and support for newcomers underscore the community's value in democratizing machine learning, making it more accessible and efficient for a broader audience. // Bio Ketan Umare is the CEO and co-founder at Union.ai. Previously he had multiple Senior roles at Lyft, Oracle, and Amazon ranging from Cloud, Distributed storage, Mapping (map-making), and machine-learning systems. He is passionate about building software that makes engineers' lives easier and provides simplified access to large-scale systems. Besides software, he is a proud father, and husband, and enjoys traveling and outdoor activities. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://union.ai/ Flyte: https://flyte.org/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/ Connect with Ketan on LinkedIn: https://www.linkedin.com/in/ketanumare/ Timestamps: [00:00] Ketan's preferred coffee [01:05] Takeaways [03:08] Please like, share, and subscribe to our MLOps channels! [03:15] Shout out to Ketan and UnionAI for sponsoring this episode! [04:23] Orchestration recent changes [07:51] Community with Flyte [11:26] ML orchestration [15:40] 50/50 is generous [20:06] Real-time ML [21:15] Over engineering without benefits [23:20] Balancing everything [27:40] Union verse Flyte [32:52] High value features of Union AI at the back of Flyte [40:18] Building LLM infrastructure [45:30] Traditional ML is the whole prompting [46:46] LLMs to evaluating prompts [48:55] Wrap up

MLOps podcast #182 with GetYourGuide's Jean Machado, DataScience Manager, Meghana Satish, MLOps Engineer, Olivia Houghton, Machine Learning Operations Engineer, Theodore Meynard, Data Science Manager, MLOps@GetYourGuide. // Abstract Join a team to talk about the journey of GYG with MLOps. From the conception of their platform to the creation of the MLOps engineer role and to their current stack state. // Bio Jean Machado Jean Carlo Machado is a DataScience Manager at GetYourGuide for the Growth Data Products team and the Machine Learning Platform Team. He is privileged to be able to work on turning ideas in data scinece from inception to production. Before GYG Jean was working in a startup in Brazil building its infrastructure from the ground up. Jean also likes community building and using technology for social good. Meghana Satish Meghana Satish is currently working as an MLOps Engineer at GetYourGuide. She has previously held positions at Amazon AWS in Berlin and Microsoft IT in Hyderabad. In addition to her career in technology, Meghana is also a talented singer, dancer, and yoga practitioner. Olivia Houghton Olivia has been working as an MLOps engineer at GetYourGuide for the past year and a half or so. Olivia's main work is in building and managing their activity ranking service. Theodore Meynard Theodore Meynard, Data Science Manager at GetYourGuide, leads the evolution of their ranking algorithm, enriching customer experiences. His hands-on journey from data scientist to leader has honed his expertise in MLOps and real-time ML. Beyond work, he's a co-organizer of PyData Berlin, underlining his commitment to community and collaborative learning. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://investing1012dot0.substack.com/ The Openness of AI report: https://research.contrary.com/reports/the-openness-of-ai ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jean on LinkedIn: https://www.linkedin.com/in/jean-carlo-machado-53b15977/ Connect with Meghana on LinkedIn: https://www.linkedin.com/in/meghana-satish-2a825282/?originalSubdomain=de Connect with Olivia on LinkedIn: https://www.linkedin.com/in/oliviaphoughton/ Connect with Theodore on LinkedIn: https://www.linkedin.com/in/theodore-meynard/

Timestamps: [00:00] GetYourGuide team's preferred coffee [00:55] Takeaways [02:20] Shout out to Berlin MLOps Community [02:38] Please like, share, and subscribe to our MLOps channels! [03:39] The GetYourGuide platform [05:45] GetYourGuide use cases [11:51] Strong Leadership Vision [13:59] Creating rituals [16:55] Feedback on the loop for improvements [18:35] Different components of GetYourGuide's ML Platform [21:04] V2 service templates [24:26] Biggest pain points [27:02] Feature flags [30:51] Data foundation [36:25] Data Testing [39:53] Cross-team Tool Adoption Process [44:59] Regrets about design decisions made in the past [47:53] What's next for the platform with LLMs? [52:49] Non-data scientists suggesting use cases, language flexibility [55:14] DevSecOps team's AI study group ideation [59:25] Experiments in growth data products, marketing split [1:01:47] Shout out to the Berlin MLOps Community! [1:03:31] Wrap up

MLOps podcast #181 with Kyle Harrison, General Partner at Contrary, The Centralization of Power in AI. // Abstract Kyle Harrison delves into the limitations imposed by language, underscoring how it can impede our grasp and manipulation of reality while stressing the critical need for improved language model performance for real-time applications. He further explores the perils of centralizing power in AI, with a specific focus on the "Openness of AI", where concerns about privacy are brought to the forefront, prompting his call for businesses to reconsider their reliance on it. The discussion also traverses the evolving landscape of AI, drawing comparisons between prominent machine learning frameworks such as TensorFlow and PyTorch. Notably, the episode underscores the vital role of open-source initiatives within the AI community and highlights the unexpected involvement of Meta in driving open-source development. // Bio Kyle Harrison is a General Partner at Contrary, where he leads Series A and growth-stage investing. He joined Contrary from Index where he was a Partner, and before that he was a growth investor at Coatue. His portfolio includes iconic startups and public companies including Ramp, Replit, Cohere, Snowflake, and Databricks. He also regularly shares his analysis on the venture capital landscape via his Substack Investing 101. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://investing1012dot0.substack.com/ The Openness of AI report: https://research.contrary.com/reports/the-openness-of-ai ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Kyle on LinkedIn: https://www.linkedin.com/in/kyle-harrison-9274b278/ Timestamps: [00:00] Kyle's preferred beverage [00:20] Takeaways [03:52] Hype in technology space [09:20] Application Layer Revenue [14:44] Stability AI Lawsuit [18:08] Concern over concentration of power in AI [20:20] Transparency concerns [23:35] Open Source AI [25:57] To use or not to use Open AI [30:51] Lack of technical expertise and business-building capabilities [35:09] AI Transparency and Accountability [37:50] Traditional ML [41:47] Finding a unique approach [45:41] AGI limitations [47:43] Using Agents [49:46] Agents getting past demos [54:39] Tech Challenges & Hoverboard Dreams [58:04] Both AI hype and skepticism are foolish [01:27] Wrap up

MLOps podcast #180 with Sachin Abeywardana Deep Learning Engineer at Canva AI, Adventures in Building CLIP and Other (Largeish) Language Models sponsored by Prem AI. // Abstract Sachin takes us on an adventure, sharing insights on the pitfalls of not understanding the broader product and the importance of incorporating AI and machine learning capabilities. From the use of AI models to grammar correction and code generation to the fascinating Clip model and the challenges of balancing work and family life, this episode promises to be both informative and thought-provoking. // Bio Sachin is the father of two beautiful children. He completed his PhD in Bayesian Machine Learning at University of Sydney in 2015. In 2016 he discovered Deep Learning and hasn't looked back. He currently works as a Senior Machine Learning Engineer at Canva and is mainly focusing on NLP problems. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Sachin Blogs: https://sachinruk.github.io/blog.html https://sachinruk.github.io/blog/ Graph ML link: http://web.stanford.edu/class/cs224w/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sachin on LinkedIn: https://www.linkedin.com/in/sachinabeywardana/ Timestamps: [00:00] Sachin's preferred beverage [00:26] Takeaways [02:30] Chat GPT user [05:58] Understanding on reliable Agents [08:10] Sachin's background [12:45] Staying at Deep Learning [16:17] Recommendation or Lead Scoring [17:36] Vector database [19:00] Sachin's blogs [23:26] The cap people [26:10] Pursuing business case [27:33] Canva [31:16] Incorporating AI and Machine Learning [32:17] Sponsor Ad [38:22] Eliminating unnecessary steps [39:00] Interacting with the product team [43:04] Criticisms on the current architecture limitations [45:58] Insufficient exploration of Transformers [47:42] Explaining GraphML [52:35] Fine-tuning ChatGPT2 [57:54] Leading ML Engineers and teams [59:40] Being practical with Math [1:05:52] Wrap up

MLOps Coffee Sessions #179 with Shahul Es, All About Evaluating LLM Applications. // Abstract Shahul Es, renowned for his expertise in the evaluation space and the creator of the Ragas Project. Shahul dives deep into the world of evaluation in open source models, sharing insights on debugging, troubleshooting, and the challenges faced when it comes to benchmarks. From the importance of custom data distributions to the role of fine-tuning in enhancing model performance, this episode is packed with valuable information for anyone interested in language models and AI. // Bio Shahul is a data science professional with 6+ years of expertise and has worked in data domains from structured, NLP to Audio processing. He is also a Kaggle GrandMaster and code owner/ ML of the Open-Assistant initiative that released some of the best open-source alternatives to ChatGPT. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links All about evaluating Large language models blog: https://explodinggradients.com/all-about-evaluating-large-language-models Ragas: https://github.com/explodinggradients/ragas --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Shahul on LinkedIn: https://www.linkedin.com/in/shahules/ Timestamps: [00:00] Shahul's preferred coffee [00:20] Takeaways [01:46] Please like, share, and subscribe to our MLOps channels! [02:07] Shahul's definition of Evaluation [03:27] Evaluation metrics and Benchmarks [05:46] Gamed leaderboards [10:13] Best at summarizing long text open-source models [11:12] Benchmarks [14:20] Recommending evaluation process [17:43] LLMs for other LLMs [20:40] Debugging failed evaluation models [24:25] Prompt injection [27:32] Alignment [32:45] Open Assist [35:51] Garbage in, garbage out [37:00] Ragas [42:52] Valuable use case besides Open AI [45:11] Fine-tuning LLMs [49:07] Connect with Shahul if you need help with Ragas @Shahules786 on Twitter [49:58] Wrap up

MLOps Coffee Sessions #178 with Stephen Batifol, Building an ML Platform: Insights, Community, and Advocacy. // Abstract Discover how Wolt onboard data scientists onto the platform and build a thriving internal community of users. Stephen's firsthand experiences shed light on the importance of developer relations and how they contribute to making data scientists' lives easier. From top-notch documentation to getting-started guides and tutorials, the internal platform at Wolt prioritizes the needs of its users. // Bio From Android developer to Data Scientist to Machine Learning Engineer, Stephen has a wealth of software engineering experience at Wolt. He believes that machine learning has a lot to learn from software engineering best practices and spends his time making ML deployments simple for other engineers. Stephen is also a founding member and organizer of the MLOps.community Meetups in Berlin. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/ Timestamps: [00:00] Stephen's preferred coffee [00:32] Takeaways [01:35] Please like, share, and subscribe to our MLOps channels! [03:00] Creating his own team! [04:44] DevRel [06:32] The door dash of Europe [11:28] Data platform underneath [12:55] Cellular core deployment uses open source [14:21] Alibi [16:08] Kafka [16:59] Selling points to data scientists [20:05] Language models concerns of data scientists [22:12] Incorporating LLMs into the business [23:55] Feedback from data scientists and end users [27:37] User surveys [30:11] Evangelizing and giving talks [35:25] Tech Hub Culture in Berlin [38:38] Kubernetes lifestyle [42:55] Interacting with SREs [45:28] Wrap up

MLOps Coffee Sessions #176 with Vin Vashishta, Collaboration and Strategy. // Abstract From the significance of technical strategists to the crucial role of product managers with a deep understanding of data and AI products, Vin shares invaluable insights on fostering collaboration, driving strategy, and maximizing the potential of data within organizations. Join us as we explore the importance of becoming multipliers in our fields, the power of effective strategy in leveraging data, and the opportunities that lie in the generative AI era. // Bio Vin's background is in applied data science. He is the founder of V Squared, one of the oldest and smallest data science consulting companies in the world. They help businesses monetize data and AI. Vin is the author of From Data To Profit. He teaches technical strategy and data and AI product management. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://www.datascience.vin/ From Data To Profit: How Businesses Leverage Data to Grow Their Top and Bottom Lines book: https://www.amazon.com/Data-Profit-Businesses-Leverage-Bottom/dp/1394196210/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vin on LinkedIn: https://www.linkedin.com/in/vineetvashishta/ Timestamps: [00:00] Vin's preferred coffee [00:14] Takeaways [02:02] Please like, share, and subscribe to our MLOps channels! [02:28] Recent ideas of Vin [05:09] Understanding the business value of any project [08:30] Generative AI making things faster [14:29] Strategy in practice [20:19] Practicality and Credibility of Strategists [22:42] Coming soon!!! LLMs in Production Conference Panel Part III [27:48] Becoming a Multiplier [29:03] The AI Product Manager [35:12] Successful monetization and integration of technologies [37:48] Justifying the ROI of LLMs [44:59] Adding that extra value [49:52] Read Vin's book linked above! [50:35] Wrap up

Sign up for our next LLM in production conference: https://go.mlops.community/prodiii

#180 with LLMs in Production Conference part 2 Ux of a LLM User Panel, Misty Free, Dina Yerlan, and Artem Harutyunyan hosted by Innovation Endeavors' Davis Treybig. // Abstract Explore different approaches to interface design, emphasizing the significance of crafting effective prompts and addressing accuracy and hallucination issues. Discover some strategies for improving latency and performance, including monitoring, scaling, and exploring emerging technologies. // Bio Misty Free Misty Free is a product manager at Jasper, where she focuses on supercharging marketers with speed and consistency in their marketing campaigns, with the power of AI. Misty has also collaborated with Stability and OpenAI to offer AI image generation within Jasper. She approaches product development with a "jobs-to-be-done" mindset, always starting with the "why" behind any need, ensuring that customer pain points are directly addressed with the features shipped at Jasper. In her free time, Misty enjoys crocheting amigurumi, practicing Spanish on Duolingo, and spending quality time with her family. Misty will be on a panel sharing her insights and experiences on the real-world use cases of LLMs. Davis Treybig Davis is a partner at Innovation Endeavors, an early-stage venture firm focused on teams solving hard technical & engineering problems. He personally focuses on computing infrastructure, AI/ML, and data. Dina Yerlan Head of Product, Generative AI Data at Adobe Firefly (family of foundation models for creatives). Artem Harutyunyan Artem is the Co-Founder & CTO at Bardeen AI. Prior to Bardeen, he was in engineering and product roles at Mesosphere and Qualys, and before that, he worked at CERN. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://www.angellist.com/venture/relay Foundation by Isaac Asimov: https://www.amazon.com/Foundation-Isaac-Asimov/dp/0553293354 AngelList Relay blog: https://www.angellist.com/blog/introducing-angellist-relay --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Davis on LinkedIn: https://www.linkedin.com/in/davistreybig/ Connect with Misty on LinkedIn: https://www.linkedin.com/in/misty-miglorin/ Connect with Dina on LinkedIn: https://www.linkedin.com/in/dinayerlan/ Connect with Artem on LinkedIn: https://www.linkedin.com/in/artemharutyunyan/

MLOps Coffee Sessions #175 with Lamia Youseff, From Virtualization to AI Integration. // Abstract Lamia discusses how both Fortune 500 companies and SMBs lack the knowledge and capabilities to identify which use cases in their systems can benefit from AI integration. She emphasizes the importance of helping these companies integrate AI effectively and acquire the necessary capabilities to stay competitive in the market. // Bio By way of an introduction, Dr. Lamia Youseff has been working in AI / ML for ~25 years, first in academia (MIT, Stanford, UCSB), then large tech (Google, Microsoft, Apple, and Facebook), and most recently with startups in Generative AI. She is currently the executive director of JazzComputing, a Visiting Research Scientist at Stanford University in Computer Science and AI, and a research affiliate with MIT Computer Science and Artificial Intelligence Lab (CSAIL). Dr. Youseff earned her Ph.D. in computer science by studying computationally intensive workloads (such as AI / ML and HPC / Scientific Codes) and has built/led several AI teams as an executive and leader at large tech companies over the years (Google, Facebook, Microsoft, and Apple). She also earned her Master's in business management, strategy, and leadership from Stanford Graduate School of Business (GSB), where she is a guest lecturer today. Dr. Youseff regularly writes and speaks about AI and Machine Learning evolution at CIO/CTO/CEO summits. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Lamia on LinkedIn: https://www.linkedin.com/in/lyouseff/ Timestamps: [00:00] Lamia's preferred coffee [01:12] Takeaways [03:00] Please like, share, and subscribe to our MLOps channels! [03:20] Lamia's background [09:52] Getting into Google Cloud [13:10] The Google Cloud project [16:38] The world before Kubernetes [19:25] Evolution of virtualization [23:20] Cloud evolution [28:13] Kubernetes enables the ecosystem [32:38] Multiple systems for machine learning [34:40] Standardization to a greater good [39:50] Complexity and pain points of ML in production [46:26] JazzComputing [50:33] Bridging gaps in AI implementation and investment [51:19] Wrap up

MLOps Coffee Sessions #178 with LLMs in Production Conference part 2 LLM on K8s Panel, Manjot Pahwa, Rahul Parundekar, and Patrick Barker hosted by Outerbounds, Inc.'s Shrinand Javadekar. // Abstract Large Language Models require a new set of tools... or do they? K8s is a beast and we like it that way. How can we best leverage all the battle-hardened tech that K8s has to offer to make sure that our LLMs go brrrrrrr. Let's talk about it in this chat. // Bio Shrinand Javadekar Shri Javadekar is currently an engineer at Outerbounds, focussed on building a fully managed, large-scale platform for running data-intensive ML/AI workloads. Earlier, he spent time trying to start an MLOps company for which he was a co-founder and head of engineering. He led the design, development, and operations of Kubernetes-based infrastructure at Intuit, running thousands of applications, built by hundreds of teams and transacting billions of $$. He has been a founding engineer of the Argo open-source project and also spent precious time at multiple startups that were acquired by large organizations like EMC/Dell and VMWare. Manjot Pahwa Manjot is an investor at Lightspeed India and focuses on SaaS and enterprise tech. She has had an operating career of over a decade within the space of fintech, SaaS, and developer tools spanning various geos such as the US, Singapore, and India. Before joining Lightspeed, Manjot headed Stripe in India, successfully obtaining the payment aggregator license, growing the team from ~10 to 100+, and driving acquisitions in the region during that time. Rahul Parundekar Rahul has 13+ years of experience building AI solutions and leading teams. He is passionate about building Artificial Intelligence (A.I.) solutions for improving the Human Experience. He is currently the founder of A.I. Hero - a platform to help you fix and enrich your data with ML. At AI Hero, he has also been a big proponent of declarative MLOps - using Kubernetes to operationalize the training and serving lifecycle of ML models and has published several tutorials on his Medium blog. Before AI Hero, he was the Director of Data Science (ML Engineering) at Figure-Eight (acquired by Appen), a data annotation company, where he built out a data pipeline and ML model serving architecture serving 36 models (NLP, Computer Vision, Audio, etc.) and traffic of up to 1M predictions per day. Patrick Barker Patrick started his career in Big Data back when that was cool, then moved into Kubernetes near its inception. He has put major features into the Kubernetes API and built several platforms on top of it. In recent years he has moved into AI, with a focus on distributed machine learning. He is now working with a startup to reshape the world of AI agents. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://www.angellist.com/venture/relay Foundation by Isaac Asimov: https://www.amazon.com/Foundation-Isaac-Asimov/dp/0553293354 AngelList Relay blog: https://www.angellist.com/blog/introducing-angellist-relay --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Shri on LinkedIn: https://www.linkedin.com/in/shrijavadekar/ Connect with Manjot on LinkedIn: https://www.linkedin.com/in/manjotpahwa/ Connect with Rahul on LinkedIn: https://www.linkedin.com/in/rparundekar/ Connect with Patrick on LinkedIn: https://www.linkedin.com/in/patrickbarkerco/

MLOps Coffee Sessions #174 with Michelle Marie Conway, Harnessing MLOps in Finance: Bringing Statistical Models to Life for Positive Impact, co-hosted by Stephen Batifol. // Abstract Michelle Marie Conway joins hosts Stephen Batifol and Demetrios to share their insights and experiences in the tech industry. Michelle emphasizes the importance of constant learning and adaptation in the rapidly changing tech industry. They discuss the need to stay up to date with the latest documentation, understand code logic, and be mindful when writing code. Michelle also reflects on their experiences as one of the few women in their university math class and often being the only woman on their team in the workplace. They discuss the need for more girls to pursue STEM subjects in schools and the importance of allies in the workplace. Additionally, Michelle explores the benefits and challenges of AI tools, sharing their experiences with tools like Gen AI and Chat GPT. While AI tools enhance productivity, Michelle also acknowledges the limitations of these tools in more technical tasks and the continued reliance on developer resources. This episode offers valuable insights into the importance of continuous learning, gender diversity in STEM, and the potential of AI tools in the field of MLOps. // Bio As an Irish woman who relocated to London after completing her university studies in Dublin, Michelle spent the past 12 years carving out a career in the data and tech industry. With a keen eye for detail and a passion for innovation, She has consistently leveraged my expertise to drive growth and deliver results for the companies she worked for. As a dynamic and driven professional, Michelle is always looking for new challenges and opportunities to learn and grow, and she's excited to see what the future holds in this exciting and ever-evolving industry. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/ Connect with Michelle on LinkedIn: https://www.linkedin.com/in/michelle-conway-40337432 Timestamps: [00:00] Michelle's preferred coffee [02:04] Takeaways [05:18] Please like, share, and subscribe to our MLOps channels! [06:18] Michelle's journey in tech [07:49] Engineering best practices [09:38] Getting comfortable with the hump [11:22] Clean coding fundamentals [13:29] Working with the people [14:09] GCP migration [18:00] GCP migration length of journey [18:38] Moving data focus [19:18] Effectiveness of running 2 systems [21:00] Dealing with discrepancies [22:15] Using Nexus [24:04] Migrating data from Teradata to BigQuery, strict security [28:48] Hiring new people [30:56] Securely managing financial data with millions of customers [32:30] When things go wrong [35:08] Finding the root cause [36:28] Dealing with the producers' problems [40:46] Rapid tech evolution constant learning [44:44] Teaching Python, using Gen AI for tasks [46:34] Dealing with LLMs use cases [49:15] Dealing with stakeholders and MLOps teams [51:17] Having a translator [52:18] Being a woman in the tech industry [55:11] Encourage more girls in STEM, support women [56:36] Women in the conversation on tech and female representation [1:03:49] Wrap up

MLOps Coffee Sessions #176 with MLOps vs. LLMOps Panel, Willem Pienaar, Chris Van Pelt, Aparna Dhinakaran, and Alex Ratner hosted by Richa Sachdev. // Abstract What do MLOps and LLMOps have in common? What has changed? Are these just new buzzwords or is there validity in calling this ops something new? // Bio Richa Sachdev A passionate and impact-driven leader whose expertise spans leading teams, architecting ML and data-intensive applications, and driving enterprise data strategy. Richa has worked for a Tier A Start-up developing feature platforms and in financial companies, leading ML Engineering teams to drive data-driven business decisions. Richa enjoys reading technical blogs focussed on system design and plays an active role in the MLOps Community. Willem Pienaar Willem is the creator of Feast, the open-source feature store and a builder in the generative AI space. Previously Willem was an engineering manager at Tecton where he led teams in both their open source and enterprise initiatives. Before that Willem built the core ML systems and created the ML platform team at Gojek, the Indonesian decacorn. Chris Van Pelt Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College. Aparna Dhinakaran Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, a pioneer and early leader in machine learning (ML) observability. A frequent speaker at top conferences and thought leader in the space, Dhinakaran was recently named to the Forbes 30 Under 30. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she built several core ML Infrastructure platforms, including Michelangelo. She has a bachelor’s from UC Berkeley's Electrical Engineering and Computer Science program, where she published research with Berkeley's AI Research group. She is on a leave of absence from the Computer Vision Ph.D. program at Cornell University. Alex Ratner Alex Ratner is the co-founder and CEO at Snorkel AI, and an Affiliate Assistant Professor of Computer Science at the University of Washington. Prior to Snorkel AI and UW, he completed his Ph.D. in CS advised by Christopher Ré at Stanford, where he started and led the Snorkel open source project, and where his research focused on defining and forwarding the concept of “data-centric AI”, the idea that labeling and developing data is the new center of the AI development workflow. His academic work focuses on data-centric AI and related topics in data management and statistical learning techniques, and applications to real-world problems in medicine, science, and more. Previously, he earned his A.B. in Physics from Harvard University. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Richa on LinkedIn: https://www.linkedin.com/in/richasachdev/ Connect with Willem on LinkedIn: https://www.linkedin.com/in/willempienaar/ Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/ Connect with Aparna on Twitter: https://www.linkedin.com/in/aparnadhinakaran/ Connect with Alex on Twitter: https://www.linkedin.com/in/alexander-ratner-038ba239/

MLOps Coffee Sessions #173 with Beyang Liu, Building Cody, an Open Source AI Coding Assistant. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Root about the development of Cody, an open-source AI coding assistant. Cody empowers developers to query and comprehend code within codebases through the integration of robust language model capabilities. Sourcegraph tackles the intricacies of understanding intricate codebases by creating comprehensive code maps and employing AI for advanced search functionalities. Cody harnesses the potential of AI to offer features such as code exploration, natural language queries, and AI-powered code generation, augmenting developer productivity and code comprehension. // Bio Beyang Liu is the CTO and Co-founder of Sourcegraph. Prior to Sourcegraph, Beyang was an engineer at Palantir Technologies building large-scale data analysis tools for Fortune 500 companies with large, complex codebases. Beyang studied computer science at Stanford, where he discovered his love for compilers and published some machine learning research as a member of the Stanford AI Lab. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://beyang.com --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Beyang on LinkedIn: https://www.linkedin.com/in/beyang-liu/ Timestamps: [00:00] Beyang's preferred coffee [00:19] Takeaways [01:25] Please like, share, and subscribe to our MLOps channels! [01:48] Beyang background before Sourcegraph [03:10] War stories [04:30] Technological tool solution [06:41] Landscape change in the past 10 years [09:32] Code search engine evolution [16:28] Vector databases [17:40] Actual tech breakdown [19:52] Incorporating AI into products amid organizational challenges [25:39] Breaking down Cody [28:04] Context fetching [30:44] AI replicating human code understanding? [36:22] Key for software creation [40:26] Speak the language [42:20] Leveraging LLMs [44:18] Low code, no code movement [47:54] Reliability issues amongst agents [53:12] LLMs used in code and chat generation [56:12] Dealing with rate limits and followers or failovers [57:33] Unnecessary comparison [1:00:26] Wrap up

MLOps Coffee Sessions #174 with Evaluation Panel, Amrutha Gujjar, Josh Tobin, and Sohini Roy hosted by Abi Aryan. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Language models are very complex thus introducing several challenges in interpretability. The large amounts of data required to train these black-box language models make it even harder to understand why a language model generates a particular output. In the past, transformer models were typically evaluated using perplexity, BLEU score, or human evaluation. However, LLMs amplify the problem even further due to their generative nature thus making them further susceptible to hallucinations and factual inaccuracies. Thus, evaluation becomes an important concern. // Bio Abi Aryan Machine Learning Engineer @ Independent Consultant Abi is a machine learning engineer and an independent consultant with over 7 years of experience in the industry using ML research and adapting it to solve real-world engineering challenges for businesses for a wide range of companies ranging from e-commerce, insurance, education and media & entertainment where she is responsible for machine learning infrastructure design and model development, integration and deployment at scale for data analysis, computer vision, audio-speech synthesis as well as natural language processing. She is also currently writing and working in autonomous agents and evaluation frameworks for large language models as a researcher at Bolkay. Amrutha Gujjar CEO & Co-Founder @ Structured Amrutha Gujjar is a senior software engineer and CEO and co-founder of Structured, based in New York. With a Bachelor of Science in Computer Science from the University of Washington's Allen School of CSE, she brings expertise in software development and leadership to my work.

Connect with Amrutha on LinkedIn to learn more about her experience and discuss exciting opportunities in software development and leadership.

Josh Tobin

Founder @ GantryJosh Tobin is the founder and CEO of Gantry. Previously, Josh worked as a deep learning & robotics researcher at OpenAI and as a management consultant at McKinsey. He is also the creator of Full Stack Deep Learning (fullstackdeeplearning.com), the first course focused on the emerging engineering discipline of production machine learning. Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel.

Sohini Roy

Senior Developer Relations Manager @ NVIDIASohini Bianka Roy is a senior developer relations manager at NVIDIA, working within the Enterprise Product group. With a passion for the intersection of machine learning and operations, Sohini specializes in the domains of MLOps and LLMOps. With her extensive experience in the field, she plays a crucial role in bridging the gap between developers and enterprise customers, ensuring smooth integration and deployment of NVIDIA's cutting-edge technologies. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Amrutha on LinkedIn: https://www.linkedin.com/in/amruthagujjar/ Connect with Josh on LinkedIn: https://www.linkedin.com/in/josh-tobin-4b3b10a9/ Connect with Sohini on Twitter: https://twitter.com/biankaroy_

MLOps Coffee Sessions #172 with Lingjiao Chen, FrugalGPT: Better Quality and Lower Cost for LLM Applications.

This episode is sponsored by QuantumBlack. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract There is a rapidly growing number of large language models (LLMs) that users can query for a fee. We review the cost associated with querying popular LLM APIs, e.g. GPT-4, ChatGPT, J1-Jumbo, and find that these models have heterogeneous pricing structures, with fees that can differ by two orders of magnitude. In particular, using LLMs on large collections of queries and text can be expensive. Motivated by this, we outline and discuss three types of strategies that users can exploit to reduce the inference cost associated with using LLMs: 1) prompt adaptation, 2) LLM approximation, and 3) LLM cascade. As an example, we propose FrugalGPT, a simple yet flexible instantiation of LLM cascade which learns which combinations of LLMs to use for different queries in order to reduce cost and improve accuracy. Our experiments show that FrugalGPT can match the performance of the best individual LLM (e.g. GPT-4) with up to 98% cost reduction or improve the accuracy over GPT-4 by 4% with the same cost. The ideas and findings presented here lay a foundation for using LLMs sustainably and efficiently. // Bio Lingjiao Chen is a Ph.D. candidate in the computer sciences department at Stanford University. He is broadly interested in machine learning, data management, and optimization. Working with Matei Zaharia and James Zou, he is currently exploring the fast-growing marketplaces of artificial intelligence and data. His work has been published at premier conferences and journals such as ICML, NeurIPS, SIGMOD, and PVLDB, and partially supported by a Google fellowship. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://lchen001.github.io/ FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance paper: https://arxiv.org/abs/2305.05176 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Lingjiao on LinkedIn: Timestamps: [00:00] Lingjiao's preferred coffee [00:35] Takeaways [02:41] Sponsor Ad: Nayur Khan of QuantumBlack [05:27] Lingjiao's research at Stanford [07:51] Day-to-day research overview [10:11] Inventing data management inspired abstractions research [13:58] Agnostic Approach to Data Management [15:56] Frugal GPT [18:59] Just another data provider [19:51] Frugal GPT breakdown [26:33] First step of optimizing the prompts [28:04] Prompt overlap [29:06] Query Concatenation [32:30] Money saving [35:04] Economizing the prompts [38:52] Questions to accommodate [41:33] LLM Cascade [47:25] Frugal GPT saves cost and Improves performance [51:37] End-user implementation [52:31] Completion Cache [56:33] Using a vector store [1:00:51] Wrap up

MLOps Coffee Sessions #172 with LLMs in Production Conference part 2 Building LLM Products Panel, George Mathew, Asmitha Rathis, Natalia Burina, and Sahar Mor Using hosted by TWIML's Sam Charrington. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract There are key areas we must be aware of when working with LLMs. High costs and low latency requirements are just the tip of the iceberg. In this panel, we hear about common pitfalls and challenges we must keep in mind when building on top of LLMs. // Bio Sam Charrington Sam is a noted ML/AI industry analyst, advisor and commentator, and host of the popular TWIML AI Podcast (formerly This Week in Machine Learning and AI). The show is one of the most popular Tech podcasts with nearly 15 million downloads. Sam has interviewed over 600 of the industry’s leading machine learning and AI experts and has conducted extensive research into enterprise AI adoption, MLOps, and other ML/AI-enabling technologies. George Mathew George is a Managing Director at Insight Partners focused on venture-stage investments in AI, ML, Analytics, and Data companies as they are establishing product/market Fit. Asmitha Rathis Asmitha is a Machine Learning Engineer with experience in developing and deploying ML models in production. She is currently working at an early-stage startup, PromptOps, where she is building conversational AI systems to assist developers. Prior to her current role, she was an ML engineer at VMware. Asmitha holds a Master’s degree in Computer Science from the University of California, San Diego, with a specialization in Machine Learning and Artificial Intelligence. Natalia Burina Natalia is an AI Product Leader who was most recently at Meta, leading Responsible AI. During her time at Meta, she led teams working on algorithmic transparency and AI Privacy. In 2017 Natalia was recognized by Business Insider as “The Most Powerful Female Engineer in 2017”. Natalia was also an Entrepreneur in Residence at Foundation Capital, advising portfolio companies and working with partners on deal flow. Prior to this, she was the Director of Product for Machine Learning at Salesforce, where she led teams building a set of AI capabilities and platform services. Prior to Facebook and Salesforce, Natalia led product development at Samsung, eBay, and Microsoft. She was also the Founder and CEO of Parable, a creative photo network bought by Samsung in 2015. Natalia started her career as a software engineer after pursuing Bachelor's degree in Applied and Computational Mathematics from the University of Washington. Sahar Mor Sahar is a Product Lead at Stripe with 15y of experience in product and engineering roles. At Stripe, he leads the adoption of LLMs and the Enhanced Issuer Network - a set of data partnerships with top banks to reduce payment fraud. Prior to Stripe he founded a document intelligence API company, was a founding PM in a couple of AI startups, including an accounting automation startup (Zeitgold, acq'd by Deel), and served in the elite intelligence unit 8200 in engineering roles. Sahar authors a weekly AI newsletter (AI Tidbits) and maintains a few open-source AI-related libraries (https://github.com/saharmor). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

MLOps Coffee Sessions #171 with Thibaut Labarre, Using Large Language Models at AngelList co-hosted by Ryan Russon. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Thibaut innovatively addressed previous system constraints, achieving scalability and cost efficiency. Leveraging AngelList investing and natural language processing expertise, they refined news article classification for investor dashboards. Central is their groundbreaking platform, AngelList Relay, automating parsing and offering vital insights to investors. Amid challenges like Azure OpenAI collaboration and rate limit solutions, Thibaut reflects candidly. The narrative highlights prompt engineering's strategic importance and empowering domain experts for ongoing advancement. // Bio Thibaut LaBarre is an engineering lead with a background in Natural Language Processing (NLP). Currently, Thibaut focuses on unlocking the potential of Large Language Model (LLM) technology at AngelList, enabling everyone within the organization to become prompt engineers on a quest to streamline and automate the infrastructure for Venture Capital. Prior to that, Thibaut began his journey at Amazon as an intern where he built Heartbeat, a state-of-the-art NLP tool that consolidates millions of data points from various feedback sources, such as product reviews, customer contacts, and social media, to provide valuable insights to global product teams. Over the span of seven years, he expanded his internship project into an organization of 20 engineers. He received a M.S. in Computational Linguistics from the University of Washington. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://www.angellist.com/venture/relay --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Ryan on LinkedIn: https://www.linkedin.com/in/ryanrusson/ Connect with Thibaut on LinkedIn: https://www.linkedin.com/in/thibautlabarre/

Timestamps: [00:00] Thibaut's preferred beverage [00:50] Takeaways [04:05] Please like, share, and subscribe to our MLOps channels! [04:44] A huge fan of Isaac Asimov [07:20] Thibaut Labarre background [09:13] AngelList as an organization [10:50] AI sense of building [12:29] System trade-offs [15:20] OpenAI's limitation [16:31] Human in the loop [17:22] Classifying relevance [18:09] Fight for value [19:37] Added value [22:10] Exploring efficient ways to automate tasks. [24:20] Investing in off-the-shelf models [27:56] AngelList Relay [30:49] News article and investment document classification technology [32:39] Back-end tech [34:09] Prompt layer [35:28] Prompt layer as a living [37:04] Foreseeing no human intervention [39:00] Blocking hallucinations [40:33] Challenges [43:49] Investments in other models besides OpenAI [45:20] Integration with other models [46:28] Ethical concerns when [48:37] OpenAI breaking Prompts [50:46] Wrap up

MLOps Coffee Sessions #170 with Phillip Carter, All the Hard Stuff with LLMs in Product Development.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract

Delve into challenges in implementing LLMs, such as security concerns and collaborative measures against attacks. Emphasize the role of ML engineers and product managers in successful implementation. Explore identifying leading indicators and measuring ROI for impactful AI initiatives. // Bio Phillip is on the product team at Honeycomb where he works on a bunch of different developer tooling things. He's an OpenTelemetry maintainer -- chances are if you've read the docs to learn how to use OTel, you've read his words. He's also Honeycomb's (accidental) prompt engineering expert by virtue of building and shipping products that use LLMs. In a past life, he worked on developer tools at Microsoft, helping bring the first cross-platform version of .NET into the world and grow to 5 million active developers. When not doing computer stuff, you'll find Phillip in the mountains riding a snowboard or backpacking in the Cascades. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://phillipcarter.dev/ https://www.honeycomb.io/blog/improving-llms-production-observability https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm https://phillipcarter.dev/posts/how-to-make-an-fsharp-code-fixer/ The "hard stuff" post: https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm Our follow-up on iterating on LLMs in prod: https://www.honeycomb.io/blog/improving-llms-production-observability --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Phillip on LinkedIn: https://www.linkedin.com/in/phillip-carter-4714a135/ Timestamps: [00:00] Phillip's preferred coffee [00:33] Takeaways [01:53] Please like, share, and subscribe to our MLOps channels! [02:45] Phillip's background [07:15] Querying Natural Language [11:25] Function calls [14:29] Pasting errors or traces [16:30] Error patterns [20:22] Honeycomb's Improvement cycle [23:20] Prompt boxes rationale [28:06] Prompt injection cycles [32:11] Injection Attempt [33:30] UI undervalued, charging the AI feature [35:11] ROI cost [44:26] Bridging ML and Product Perspective [52:53] AI Model Trade-offs [56:33] Query assistant [59:07] Honeycomb is hiring! [1:00:08] Wrap up

MLOps Coffee Sessions #169 with Barak Turovsky, MLOps at the Age of Generative AI.

Thanks to wandb.ai for sponsoring this episode. Check out their new course on evaluating and fine-tuning LLMs wandb.me/genai-mlops.course // Abstract The talk focuses on MLOps aspects of developing, training and serving Generative AI/Large Language models // Bio Barak is an Executive in Residence at Scale Venture Partners, a leading Enterprise venture capital firm. Barak spent 10 years as Head of Product and User Experience for Languages AI and Google Translate teams within the Google AI org, focusing on applying cutting-edge Artificial Intelligence and Machine Learning technologies to deliver magical experiences across Google Search, Assistant, Cloud, Chrome, Ads, and other products. Previously, Barak spent 2 years as a product leader within the Google Commerce team. Most recently, Barak served as Chief Product Officer, responsible for product management and engineering at Trax, a leading provider of Computer Vision AI solutions for Retail and Commerce industries. Prior to joining Google in 2011, Barak was Director of Products in Microsoft’s Mobile Advertising, Head of Mobile Commerce at PayPal, and Chief Technical Officer at an Israeli start-up. He lived more than 10 years in 3 different countries (Russia, Israel, and the US) and fluently speaks three languages. Barak earned a Bachelor of Laws degree from Tel Aviv University, Israel, and a Master’s of Business Administration from the University of California, Berkeley. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Bio and links about Barak's work: https://docs.google.com/document/d/1E4Yrmt_Y57oTEYHQQDvt71XzSJ8Ew5WvscAQbHV4K3U/edit Framework for evaluating Generative AI use cases: https://www.linkedin.com/pulse/framework-evaluating-generative-ai-use-cases-barak-turovsky/?trackingId=%2BMRxEZ9WTPCNH2JscILTeg%3D%3D The Great A.I. Awakening: https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Barak on LinkedIn: https://www.linkedin.com/in/baraktur/ Timestamps: [00:00] Barak's preferred coffee [00:23] Barak Turovsky's background [03:10] Please like, share, and subscribe to our MLOps channels! [04:09] Getting into tech [08:39] First wave of AI [12:39] Building a product at a scale and the challenges [15:59] Framework for evaluating Generative AI use cases [24:33] Machine trust adoption [29:13] Wandb's new course [31:10] Focus on achievable use cases for LLMs. [36:36] User feedback [38:23] Disruption of entertainment and customer interactions [46:14] Get new tools or work with your own distribution? [47:57] Importance of data engineers [53:28] ML Engineers Collaborate with Product [56:13] Wrap up

MLOps Coffee Sessions #168 with Piotr Niedźwiedź, Experiment Tracking in the Age of LLMs, co-hosted by Vishnu Rachakonda. // Abstract Piotr shares his journey as an entrepreneur and the importance of focusing on core values to achieve success. He highlights the mission of Neptune to support ML teams by providing them with control and confidence in their models. The conversation delves into the role of experiment tracking in understanding and debugging models, comparing experiments, and versioning models. Piotr introduces the concept of prompt engineering as a different approach to building models, emphasizing the need for prompt validation and testing methods. // Bio Piotr is the CEO of neptune.ai. Day to day, apart from running the company, he focuses on the product side of things. Strategy, planning, ideation, getting deep into user needs and use cases. He really likes it.
Piotr's path to ML started with software engineering. Always liked math and started programming when he was 7. In high school, Piotr got into algorithmics and programming competitions and loved competing with the best. That got him into the best CS and Maths program in Poland which funny enough today specializes in machine learning.
Piotr did his internships at Facebook and Google and was offered to stay in the Valley. But something about being a FAANG engineer didn’t feel right. He had this spark to do more, build something himself. So with a few of his friends from the algo days, they started Codilime, a software consultancy, and later a sister company Deepsense.ai machine learning consultancy, where he was a CTO.
When he came to the ML space from software engineering, he was surprised by the messy experimentation practices, lack of control over model building, and a missing ecosystem of tools to help people deliver models confidently.
It was a stark contrast to the software development ecosystem, where you have mature tools for DevOps, observability, or orchestration to execute efficiently in production. And then, one day, some ML engineers from Deepsense.ai came to him and showed him this tool for tracking experiments they built during a Kaggle competition (which they won btw), and he knew this could be big.
He asked around, and everyone was struggling with managing experiments. He decided to spin it off as a VC-funded product company, and the rest is history. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://neptune.ai/blog/author/piotr-niedzwiedz

https://www.youtube.com/playlist?list=PLKePQLVx9tOfKFbg9GY2Anl41Be4T1-m5 
https://thesequence.substack.com/p/-piotr-niedzwiedz-neptunes-ceo-on

https://open.spotify.com/episode/2KEqTMAHODbPKdUEtlrhm7?si=ed862b2ac7534e39
https://www.linkedin.com/in/piotrniedzwiedz/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Piotr on LinkedIn: https://www.linkedin.com/in/piotrniedzwiedz/ Timestamps: [00:00] Introduction to Piotr Niedźwiedź [01:35] Please like, share, and subscribe to our MLOps channels! [01:58] Wojciech Zaremba [05:20] The Olympiad [06:31] Building own company [12:16] Talking outside Poland with the same passion [13:45] Adapting with Neptune [19:35] Core values focus [22:02] Product Vision changes with advances [29:36] Control and confidence [30:05] Experiment tracking existing use cases [37:25] Control pane [38:59] Piotr's prediction [43:20] WiFi issues around the world [44:09] Wrap up

MLOps Coffee Sessions #167 with Maxime Beauchemin, Treating Prompt Engineering More Like Code. // Abstract Promptimize is an innovative tool designed to scientifically evaluate the effectiveness of prompts. Discover the advantages of open-sourcing the tool and its relevance, drawing parallels with test suites in software engineering. Uncover the increasing interest in this domain and the necessity for transparent interactions with language models. Delve into the world of prompt optimization, deterministic evaluation, and the unique challenges in AI prompt engineering. // Bio Maxime Beauchemin is the founder and CEO of Preset, a series B startup supporting and commercializing the Apache Superset project. Max was the original creator of Apache Airflow and Apache Superset when he was at Airbnb. Max has over a decade of experience in data engineering, at companies like Lyft, Airbnb, Facebook, and Ubisoft. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Max's first MLOps Podcast episode: https://go.mlops.community/KBnOgN Test-Driven Prompt Engineering for LLMs with Promptimize blog: https://maximebeauchemin.medium.com/mastering-ai-powered-product-development-introducing-promptimize-for-test-driven-prompt-bffbbca91535https://maximebeauchemin.medium.com/mastering-ai-powered-product-development-Test-Driven Prompt Engineering for LLMs with Promptimize podcast: https://talkpython.fm/episodes/show/417/test-driven-prompt-engineering-for-llms-with-promptimizeTaming AI Product Development Through Test-driven Prompt Engineering // Maxime Beauchemin // LLMs in Production Conference lightning talk: https://home.mlops.community/home/videos/taming-ai-product-development-through-test-driven-prompt-engineering --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Max on LinkedIn: https://www.linkedin.com/in/maximebeauchemin/ Timestamps: [00:00] Max introducing the Apache Superset project at Preset [01:04] Max's preferred coffee [01:16] Airflow creator [01:45] Takeaways [03:53] Please like, share, and subscribe to our MLOps channels! [04:31] Check Max's first MLOps Podcast episode [05:20] Promptimize [06:10] Interaction with API [08:27] Deterministic evaluation of SQL queries and AI [12:40] Figuring out the right edge cases [14:17] Reaction with Vector Database [15:55] Promptomize Test Suite [18:48] Promptimize vision [20:47] The open-source blood [23:04] Impact of open source [23:18] Dangers of open source [25:25] AI-Language Models Revolution [27:36] Test-driven design [29:46] Prompt tracking [33:41] Building Test Suites as Assets [36:49] Adding new prompt cases to new capabilities [39:32] Monitoring speed and cost [44:07] Creating own benchmarks [46:19] AI feature adding more value to the end users [49:39] Perceived value of the feature [50:53] LLMs costs [52:15] Specialized model versus Generalized model [56:58] Fine-tuning LLMs use cases [1:02:30] Classic Engineer's Dilemma [1:03:46] Build exciting tech that's available [1:05:02] Catastrophic forgetting [1:10:28] Promt driven development [1:13:23] Wrap up

MLOps Coffee Sessions #166 with Roy Hasson & Santona Tuli, Eliminating Garbage In/Garbage Out for Analytics and ML. // Abstract Shift left data quality ownership and observability that makes it easy for users to catch bad data at the source and stop it from entering your analytics/ML stack. // Bio Santona Tuli Santona Tuli, Ph.D. began her data journey through fundamental physics—searching through massive event data from particle collisions at CERN to detect rare particles. She’s since extended her machine learning engineering to natural language processing, before switching focus to product and data engineering for data workflow authoring frameworks. As a Python engineer, she started with the programmatic data orchestration tool, Airflow, helping improve its developer experience for data science and machine learning pipelines. Currently, at Upsolver, she leads data engineering and science, driving developer research and engagement for the declarative workflow authoring framework in SQL. Dr. Tuli is passionate about building, as well as empowering others to build, end-to-end data and ML pipelines, scalably. Roy Hasson Roy is the head of product at Upsolver helping companies deliver high-quality data to their analytics and ML tools. Previously, Roy led product management for AWS Glue and AWS Lake Formation. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://royondata.substack.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Roy on LinkedIn: https://www.linkedin.com/in/royhasson/ Connect with Santona on LinkedIn: https://www.linkedin.com/in/santona-tuli/ Timestamps: [00:00] Santona's and Roy's preferred coffee [01:05] Santona's and Roy's background [03:33] Takeaways [05:49] Please like, share, and subscribe to our MLOps channels! [06:42] Back story of having Santona and Roy on the podcast [09:51] Santona's story [11:37] Optimal tag teamwork [16:53] Dealing with stakeholder needs [26:25] Having mechanisms in place [27:30] Building for data Engineers vs building for data scientists [34:50] Creating solutions for users [38:55] User experience holistic point of view [41:11] Tooling sprawl is real [42:00] LLMs reliability [45:00] Things would have loved to learn five years ago [49:46] Wrap up

MLOps Coffee Sessions #165 with Sammy Sidhu, Python Power: How Daft Embeds Models and Revolutionizes Data Processing. // Abstract Sammy shares his fascinating journey in the autonomous vehicle industry, highlighting his involvement in two successful startup acquisitions by Tesla and Toyota. He emphasizes his expertise in optimizing and distilling models for efficient machine learning, which he has incorporated into his new company Eventual. The company's open-source offering, daf, focuses on tackling the challenges of unstructured and complex data. Sammy discusses the future of MLOps, machine learning, and data storage, particularly in relation to the retrieval and processing of unstructured data. The Eventual team is developing Daft, an open-source query engine that aims to provide efficient data storage solutions for unstructured data, offering features like governance, schema evolution, and time travel. The conversation sheds light on the innovative developments in the field and the potential impact on various industries. // Bio Sammy is a Deep Learning and systems veteran, holding over a dozen publications and patents in the space. Sammy graduated from the University of California, Berkeley where he did research in Deep Learning and High Performance Computing. He then joined DeepScale as the Chief Architect and led the development of perception technologies for autonomous vehicles. During this time, DeepScale grew rapidly and was subsequently acquired by Tesla in 2019. Staying in Autonomous Vehicles, Sammy joined Lyft Level 5 as a Senior Staff Software Engineer, building out core perception algorithms as well as infrastructure for machine learning and embedded systems. Level 5 was then acquired by Toyota in 2021, adopting much of his work. Sammy is now CEO and Co-Founder at Eventual Building Daft, an open-source query engine that specializes in multimodal data. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://sammysidhu.com/ Check out Daft, our open-source query engine for multimodal data! https://www.getdaft.io/ Here are some talks/shows we have given about it: - PyData Global (Dec 2022): Large-scale image processing: https://www.youtube.com/watch?v=ol6IQUbyeDo&ab_channel=PyData - Ray Meetup (March 2023): Distributed ML preprocessing + training on Ray https://www.youtube.com/watch?v=1MpEYlIlu7w&t=2972s&ab_channel=Anyscale - The Data Stack Show (April 2023): Self-Driving Technology and Data Infrastructure with Sammy Sidhu https://datastackshow.com/podcast/the-prql-self-driving-technology-and-data-infrastructure-with-sammy-sidhu-co-founder-and-ceo-of-eventual/ Chain of Thought for LLMs: https://cobusgreyling.medium.com/chain-of-thought-prompting-in-llms-1077164edf97 Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes: https://arxiv.org/abs/2305.02301 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sammy on LinkedIn: https://www.linkedin.com/in/sammy-sidhu/

MLOps Coffee Sessions #164 with Rob Hirschfeld, Open Source and Fast Decision Making. This episode is brought to you by. // Abstract Rob Hirschfeld, the CEO and co-founder of Rack N, discusses his extensive experience in the DevOps movement. He shares his notable achievement of coining the term "the cloud" and obtaining patents for infrastructure management and API provision. Rob highlights the stagnant progress in operations and the persistent challenges in security and access controls within the industry. The absence of standardization in areas such as Kubernetes and single sign-on complicates the development of robust solutions. To address these issues, Rob underscores the significance of open-source practices, automation, and version control in achieving operational independence and resilience in infrastructure management. // Bio Rob is the CEO and Co-founder of RackN, an Austin-based start-up that develops software to help automate data centers, which they call Digital Rebar. This platform helps connect all the different pieces and tools that people use to manage infrastructure into workflow pipelines through seamless multi-component automation across the different pieces and parts needed to bring up IT systems, platforms, and applications. Rob has a background in Scale Computing, Mechanical and Systems Engineering, and specializes in large-scale complex systems that are integrated with the physical environment. He has founded companies and been in the cloud and infrastructure space for nearly 25 years and has done everything from building the first Clouds using ESXi betas to serving four terms on the OpenStack Foundation Board. Rob was trained as an Industrial Engineer and holds degrees from Duke University and Louisiana State University. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://rackn.com/ https://robhirschfeld.com/about/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Rob on LinkedIn: https://www.linkedin.com/in/rhirschfeld/ Timestamps: [00:00] Rob's preferred coffee [00:17] Rob Hirschfeld's background [01:42] Takeaways [02:36] Please like, share, and subscribe to this channel! [03:09] Creation of Cloud [08:38] Changes in Cloud after 25 Years [10:54] Pros and cons of microservices [13:06] Secure Access Provisioning [15:46] Parallelism with ads [18:08] Redfish protocol [20:21] Impact of using open source vs using a SAS provider [26:15] Automation [32:39] Embrace Operational Flexibility [35:08] Automating infrastructure inefficiently [41:26] Legacy code and resiliency [43:39] Collection of metadata [45:50] RackN [51:23] Granular Cloud Preferences [54:35] Reframing of perceived complexity [57:32] Generative DevOps [58:50] Wrap up

MLOps Coffee Sessions #163 with Yujian Tang, Democratizing AI co-hosted by Abi Aryan. // Abstract The popularity of ChatGPT has brought large language model (LLM) apps and their supporting technologies to the forefront. One of the supporting technologies is vector databases. Yujian shares how vector databases like Milvus are used in production and how they solve one of the biggest problems in LLM app building - data issues. They also discuss how Zilliz is democratizing vector databases through education, expanding access to technologies, and technical evangelism. // Bio Yujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published to conferences including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Yujian on LinkedIn: https://www.linkedin.com/in/yujiantang Timestamps: [00:00] Yujian's preferred coffee [02:40] Takeaways [05:14] Please share this episode with your friends! [06:39] Vector databases trajectory [09:00] 2 start-up companies created by Yujian [09:39] Uninitiated Vector Databases [12:20] Vector Databases trade-off [14:16] Difficulties in training LLMs [23:30] Enterprise use cases [27:38] Process/rules not to use LLMs unless necessary [32:14] Setting up returns [33:13] When not to use Vector Databases [35:30] Elastic search [36:07] Generative AI apps common pitfalls [39:35] Knowing your data [41:50] Milvus [48:28] Actual Enterprise use cases [49:32] Horror stories [50:31] Data mesh [51:06] GPTCash [52:10] Shout out to the Seattle Community! [53:44] Wrap up

MLOps Coffee Sessions #162 with Soham Chatterjee, From LLMs to TinyML: The Dynamic Spectrum of MLOps co-hosted by Abi Aryan. // Abstract Explore the spectrum of MLOps from large language models (LLMs) to TinyML. Soham highlights the difficulties of scaling machine learning models and cautions against relying exclusively on open AI's API due to its limitations. Soham is particularly interested in the effective deployment of models and the integration of IoT with deep learning. He offers insights into the challenges and strategies involved in deploying models in constrained environments, such as remote areas with limited power and utilizing small devices like Arduino Nano. // Bio Soham leads the machine learning team at Sleek, where he builds tools for automated accounting and back-office management. As an electrical engineer, Soham has a passion for the intersection of machine learning and electronics, specifically TinyML/Edge Computing. He has several courses on MLOps and TinyMLOps available on Udacity and LinkedIn, with more courses in the works. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Soham on LinkedIn: https://www.linkedin.com/in/soham-chatterjee Timestamps: [00:00] Soham's preferred coffee [01:49] Takeaways [05:33] Please share this episode with [07:02] Soham's background [09:00] From electrical engineering to Machine Learning [10:40] Deep learning, Edge Computing, and Quantum Computing [11:34] Tiny ML [13:29] Favorite area in Tiny ML chain [14:03] Applications explored [16:56] Operational challenges transformation [18:49] Building with Large Language Models [25:44] Most Optimal Model [26:33] LLMs path [29:19] Prompt engineering [33:17] Migrating infrastructures to new product [37:20] Your success where others failed [38:26] API Accessibility [39:02] Reality about LLMs [40:39] Compression angle adds to the bias [43:28] Wrap up

MLOps Coffee Sessions #161 with Tuhin Srivastava, The Long Tail of ML Deployment co-hosted by Abi Aryan. This episode is brought to you by QuantumBlack. // Abstract Baseten is an engineer-first platform designed to alleviate the engineering burden for machine learning and data engineers. Tuhin's perspective, based on research with Stanford students, emphasizes the importance of engineers embracing the engineering aspects and considering them from a reproductive perspective. // Bio Tuhin Srivastava is the co-founder and CEO of Baseten. Tuhin has spent the better part of the last decade building machine learning-powered products and is currently working on empowering engineers to build production-grade services with machine learning. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links QuantumBlack: https://www.mckinsey.com/capabilities/quantumblack/contact-us

Baseten: https://www.baseten.co/ Baseten Careers: https://www.baseten.co/careers --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Tuhin on LinkedIn: https://www.linkedin.com/in/tuhin-srivastava-60601114/ Timestamps: [00:00] Partnership with QuantumBlack [00:16] Nayur Khan presenting QuantumBlack [03:35] QuantumBlack is hiring! [03:47] Tuhin's preferred coffee [05:03] Takeaways [07:00] Please share this episode with a friend! [07:12] Comments/Reviews [08:49] Tuhin's background [10:13] Finance and Law common complaint culture [11:40] Doing Machine Learning in 2010 - 2011 [14:31] Gum broad or the next company shape? [16:33] Engineers need to learn machine learning [20:18] Software engineers need to dig deeper [24:49] Cambrian Explosion [27:53] The Holy Trifecta [28:54] Objective truth and prompting [31:23] Limitations of LLMs [35:37] Documentation challenges [38:25] Baseten creating valuable models [40:37] Advocate for Microservices or API-based solution [42:54] Learning Git pains [44:16] Baseten back ups [48:00] Baseten is hiring! [49:32] Wrap up

MLOps Coffee Sessions #160 with Matt Sharp, Data Developer at Shopify, Clean Code for Data Scientists co-hosted by Abi Aryan. // Abstract Let's delve into Shopify's real-time serving platform, Merlin, which enables features like recommender systems, inbox classification, and fraud detection. Matt shares his insights on clean coding and the new book he is writing about LLMs in production. // Bio Matt is a Chemical Engineer turned Data scientist turned Data Engineer. Self-described "Recovering Data Scientist", Matt got tired of all the inefficiencies he faced as a Data Scientist and made the switch to Data Engineering. At Matt's last job, he ended up building the entire MLOps platform from scratch for a fintech startup called MX. Matt gives tips to data scientists on LinkedIn on how to level up their careers and has started to be known for my clean code tips in particular. Matt recently just started a new job at Shopify. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewsharp/

MLOps Coffee Sessions #159 with Maria Vechtomova, Lead ML engineer, and Basak Eskili Machine Learning Engineer, at Ahold Delhaize, Why is MLOps Hard in an Enterprise? co-hosted by Abi Aryan. // Abstract MLOps is particularly challenging to implement in enterprise organizations due to the complexity of the data ecosystem, the need for collaboration across multiple teams, and the lack of standardization in ML tooling and infrastructure. In addition to these challenges, at Ahold Delhaize, there is a requirement for the reusability of models as our brands seek to have similar data science products, such as personalized offers, demand forecasts, and cross-sell. // Bio Maria Vechtomova Maria is a Machine Learning Engineer at Ahold Delhaize. Maria is bridging the gap between data scientists infra and IT teams at different brands and focuses on standardization of machine learning operations across all the brands within Ahold Delhaize. During nine years in Data&Analytics, Maria tried herself in different roles, from data scientist to machine learning engineer, was part of teams in various domains, and has built broad knowledge. Maria believes that a model only starts living when it is in production. For this reason, last six years, her focus was on the automation and standardization of processes related to machine learning. Basak Eskili Basak Eskili is a Machine Learning Engineer at Ahold Delhaize. She is working on creating new tools and infrastructure that enable data scientists to quickly operationalise algorithms. She is bridging the space between data scientists and platform engineers while improving the way of working in accordance with MLOps principles. In her previous role, she was responsible for bringing models to production. She focused on NLP projects and building data processing pipelines. Basak also implemented new solutions by using cloud services for existing applications and databases to improve time and efficiency. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links MLOps Maturity Assessment Blog: https://mlops.community/mlops-maturity-assessment/ The Minimum Set of Must-Haves for MLOps Blog: https://mlops.community/the-minimum-set-of-must-haves-for-mlops/ Traceability & Reproducibility Blog: https://mlops.community/traceability-reproducibility/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Maria on LinkedIn: https://www.linkedin.com/in/maria-vechtomova/Connect with Basak on LinkedIn: https://www.linkedin.com/in/ba%C5%9Fak-tu%C4%9F%C3%A7e-eskili-61511b58/

MLOps Coffee Sessions #158 with Nils Reimer, MLOps Build or Buy, Large Language Model at Scale co-hosted by Abi Aryan. // Abstract Large Language Models with billions of parameters have the possibility to change how we work with textual data. However, running them on scale at potentially hundred millions of texts a day is a massive challenge. Nils talks about finding the right model size for respective tasks, model distillation, and promising new ways on transferring knowledge from large to smaller models. // Bio Nils Reimers is highly recognized throughout the AI community for creating and maintaining the now-famous Sentence Transformers library (www.SBERT.net) used to develop, train, and use state-of-the-art LLMs. The project has 900+ stars on GitHub and 30M+ installations. Nils is currently the Director of Machine Learning at Cohere where he leads the team that develops and trains Large Language Models (LLM) with billions of parameters. Prior to Cohere, Nils created and led the science team for Neural Search at HuggingFace. Nils holds a Ph.D. in Computer Science from UKP in Darmstadt. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links (www.SBERT.net) https://www.nils-reimers.de/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Nils on LinkedIn: https://www.linkedin.com/in/reimersnils/ Timestamps: [00:00] Nils' preferred coffee [00:45] Nils' background [01:30] Takeaways [06:47] Subscribe to our Newsletters and IRL Meetups, and leave your reviews! [07:32] Nils' history [10:39] From IT Security to Machine Learning [13:22] Tangibility of IT and Security [14:46] NLP transition [15:55] Bad augmentation to new capabilities of LLMs [16:59] Nils' concern during his PH.D. [19:55] Making Money from Machine Learning [22:06] Catastrophic forgetting [26:34] Updating solutions [28:42] Neural search space and building adaptive models [31:23] Filtering models [32:32] Latency issues [36:53] Models running in parallel [37:54] Generative models problems [38:43] Nils' role at Cohere [41:41] To build or not to build API [43:00] Search models [45:38] Large use cases [46:43] Open source discussion within Cohere [50:48] Competitive Edge [55:27] Future world of API [58:14] LLMs in Production Conference Part 2 announcement! [1:00:17] Hopeful direction of Cohere's future [1:02:33] Data silos [1:04:34] Where to update the database and code [1:05:24] Nils' focus [1:08:49] Make money or save money [1:10:30] Cohere's upcoming project [1:12:37] Time spent red teaming the models [1:14:05] Wrap up

We are having another LLMs in-production Virtual Conference. 50+ speakers combined with in-person activities around the world on June 15 & 16. Sign up free here: https://home.mlops.community/home/events/llm-in-prod-part-ii-2023-06-20 // Abstract This panel discussion is centered around a crucial topic in the tech industry - data privacy and security in the context of large language models and AI systems. The discussion highlights several key themes, such as the significance of trust in AI systems, the potential risks of hallucinations, and the differences between low and high-affordability use cases. The discussion promises to be thought-provoking and informative, shedding light on the latest developments and concerns in the field. We can expect to gain valuable insights into an issue that is becoming increasingly relevant in our digital world. // Bio Diego Oppenheimer Diego Oppenheimer is an entrepreneur, product developer, and investor with an extensive background in all things data. Currently, he is a Partner at Factory a venture fund specializing in AI investments as well as interim head of product at two LLM startups. Previously he was an executive vice president at DataRobot, Founder, and CEO at Algorithmia (acquired by DataRobot), and shipped some of Microsoft’s most used data analysis products including Excel, PowerBI, and SQL Server. Diego is active in AI/ML communities as a founding member and strategic advisor for the AI Infrastructure Alliance and MLops.Community and works with leaders to define ML industry standards and best practices. Diego holds a Bachelor's degree in Information Systems and a Masters degree in Business Intelligence and Data Analytics from Carnegie Mellon University Gevorg Karapetyan Gevorg Karapetyan is the co-founder and CTO of ZERO Systems where he oversees the company's product and technology strategy. He holds a Ph.D. in Computer Science and is the author of multiple publications, including a US Patent. Vin Vashishta C-level Technical Strategy Advisor and Founder of V Squared, one of the first data science consulting firms. Our mission is to provide support and clarity for our clients’ complete data and AI monetization journeys. Over a decade in data science and a quarter century in technology building and leading teams and delivering products with $100M+ in ARR. Saahil Jain Saahil Jain is an engineering manager at You.com. At You.com, Saahil builds search, ranking, and conversational AI systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science at Columbia University and Stanford University respectively. Shreya Rajpal Shreya is the creator of Guardrails AI, an open-source solution designed to establish guardrails for large language models. As a founding engineer at Predibase, she helped build the Applied ML and ML infra teams. Previously, she worked at Apple's Special Projects Group on cross-functional ML, and at Drive.ai building computer vision models.

MLOps Coffee Sessions #157 with Katrina Ni & Aaron Maurer, MLOps Build or Buy, Startup vs. Enterprise? co-hosted by Jake Noble of tecton.ai.

This episode is sponsored by tecton.ai - Check out their feature store to get your real-time ML journey started. // Abstract There are a bunch of challenges with building useful machine learning at a B2B software company like Slack, but we've built some cool use cases over the years, particularly around recommendations. One of the key challenges is how to train powerful models while being prudent stewards of our clients' essential business data, and how to do so while respecting the increasingly complex landscape of international data regulation. // Bio Katrina Ni Katrina is a Machine Learning Engineer in Slack ML Services Team where they build ML platforms and integrate ML, e.g. Recommend API, Spam Detection, across product functionalities. Prior to Slack, she is a Software Engineer in Tableau Explain Data Team where they build tools that utilize statistical models and propose possible explanations to help users inspect, uncover, and dig deeper into the viz. Aaron Maurer Aaron is a senior engineering manager in the infra organization at Slack, managing both the machine learning team and the real-time services team. In six years at Slack, most of which Aaron spent as an engineer, He worked on the search ranking, recommendation, spam detection, performance anomaly detection, and many other ML applications. Aaron is also an advisor to Eppo, an experimentation platform. Prior to Slack, Aaroon worked as a data scientist at Airbnb, earned a Masters in statistics at the University of Chicago, and helped develop econometric models projecting the Obamacare rollout at Acumen LLC. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jake on LinkedIn: https://www.linkedin.com/in/jakednoble/ Connect with Katrina on LinkedIn: https://www.linkedin.com/in/katrina-ni-660b2590/ Connect with Aaron on LinkedIn: https://www.linkedin.com/in/aaron-maurer-4003b638/ Timestamps: [00:00] Aaron and Katrina's preferred coffee [00:41] Recommender and System and Jake [02:06] Takeaways [05:38] Introduction to Aaron Maurer & Katrina Ni [06:53] Aaron Maurer & Katrina Ni's Recommend API blog post [08:36] 10-pole machine learning use case and Rex's use case [10:14] Genesis of Slack's recommender system framework [11:47] The Special Sauce [12:58] Speaking the same language [15:23] Use case sources [17:08] Slack's feature engineering [17:52] Main CTR models [18:40] Data privacy [21:33] Slack's recommendations problem [22:09] Fine-tuning the generative models [23:30] Cold start problem [26:02] Underrated [28:24] Baseline [28:55] Cold sore space [30:15] LLMs in Production Conference Part 2 announcement! [32:32] Data scientists transition to ML [33:35] Unicorns do exist! [34:43] Diversity of skill set [36:02] The future of ML [38:34] Model Serving [40:11] MLOps Maturity level [43:06] AWS Analogy [45:05] Primary difficulty [48:07] Wrap up

Sign up for the next LLM in production conference here: https://go.mlops.community/LLMinprod

Watch all the talks from the first conference: https://go.mlops.community/llmconfpart1

// Abstract In this panel discussion, the topic of the cost of running large language models (LLMs) is explored, along with potential solutions. The benefits of bringing LLMs in-house, such as latency optimization and greater control, are also discussed. The panelists explore methods such as structured pruning and knowledge distillation for optimizing LLMs. OctoML's platform is mentioned as a tool for the automatic deployment of custom models and for selecting the most appropriate hardware for them. Overall, the discussion provides insights into the challenges of managing LLMs and potential strategies for overcoming them. // Bio Lina Weichbrodt Lina is a pragmatic freelancer and machine learning consultant that likes to solve business problems end-to-end and make machine learning or a simple, fast heuristic work in the real world. In her spare time, Lina likes to exchange with other people on how they can implement best practices in machine learning, talk to her at the Machine Learning Ops Slack: shorturl.at/swxIN. Luis Ceze Luis Ceze is Co-Founder and CEO of OctoML, which enables businesses to seamlessly deploy ML models to production making the most out of the hardware. OctoML is backed by Tiger Global, Addition, Amplify Partners, and Madrona Venture Group. Ceze is the Lazowska Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he has taught for 15 years. Luis co-directs the Systems and Architectures for Machine Learning lab (sampl.ai), which co-authored Apache TVM, a leading open-source ML stack for performance and portability that is used in widely deployed AI applications. Luis is also co-director of the Molecular Information Systems Lab (misl.bio), which led pioneering research in the intersection of computing and biology for IT applications such as DNA data storage. His research has been featured prominently in the media including New York Times, Popular Science, MIT Technology Review, and the Wall Street Journal. Ceze is a Venture Partner at Madrona Venture Group and leads their technical advisory board. Jared Zoneraich Co-Founder of PromptLayer, enabling data-driven prompt engineering. Compulsive builder. Jersey native, with a brief stint in California (UC Berkeley '20) and now residing in NYC. Daniel Campos Hailing from Mexico Daniel started his NLP journey with his BS in CS from RPI. He then worked at Microsoft on Ranking at Bing with LLM(back when they had 2 commas) and helped build out popular datasets like MSMARCO and TREC Deep Learning. While at Microsoft he got his MS in Computational Linguistics from the University of Washington with a focus on Curriculum Learning for Language Models. Most recently, he has been pursuing his Ph.D. at the University of Illinois Urbana Champaign focusing on efficient inference for LLMs and robust dense retrieval. During his Ph.D., he worked for companies like Neural Magic, Walmart, Qualtrics, and Mendel.AI and now works on bringing LLMs to search at Neeva. Mario Kostelac Currently building AI-powered products in Intercom in a small, highly effective team. I roam between practical research and engineering but lean more towards engineering and challenges around running reliable, safe, and predictable ML systems. You can imagine how fun it is in LLM era :). Generally interested in the intersection of product and tech, and building a differentiation by solving hard challenges (technical or non-technical). Software engineer turned into Machine Learning engineer 5 years ago.

MLOps Coffee Sessions #156 with Melissa Barr & Michael Mui, Machine Learning Education at Uber co-hosted by Lina Weichbrodt. // Abstract Melissa and Michael discuss the education program they developed for Uber's machine learning platform service, Michelangelo, during a guest appearance on a podcast. The program teaches employees how to use machine learning both in general and specifically for Uber. The platform team can obtain valuable feedback from users and use it to enhance the platform. The course was designed using engineering principles, making it applicable to other products as well. // Bio Melissa Barr Melissa is a Technical Program Manager for ML & AI at Uber. She is based in New York City. She drives projects across Uber’s ML platform, delivery, and personalization teams. She also built out the first version of the ML Education Program in 2021. Michael Mui Melissa is a Staff Technical Lead Manager on Uber AI's Machine Learning Platform team. He leads the Distributed ML Training team which focuses on building elastic, scalable, and fault-tolerant distributed machine learning libraries and systems used to power machine learning development productivity across Uber. He also co-leads Uber’s internal ML Education initiatives. Outside of Uber, Michael also teaches ML at the Parsons School of Design in NYC as an Adjunct Faculty (mostly for the museum passes!) and guest lectures at the University of California, Berkeley. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.uber.com/blog/ml-education-at-uber-program-design-and-outcomes/https://www.uber.com/blog/ml-education-at-uber/https://www.uber.com/en-PH/blog/ml-education-at-uber/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Melissa on LinkedIn: https://www.linkedin.com/in/melissabarr1/ Connect with Michael on LinkedIn: https://www.linkedin.com/in/michael-c-mui/Connect with Lina on LinkedIn: https://www.linkedin.com/in/lina-weichbrodt-344a066a/

Timestamps: [00:00] Melissa and Michael's preferred coffee [01:51] Takeaways [05:40] Please subscribe to our newsletters and leave reviews on our podcasts! [06:18] Machine learning at Uber education program [07:45] The Uber courses [10:03] Tailoring the Uber education system [12:27] Growing out of the ML-Ed platform efforts [14:14] Expanding the ML Market Size [15:23] Relationship evolution [17:36] Reproducibility best practices [21:46] Learning development timeline [26:29] Courses effectiveness evaluation [29:57] Tracking Progress Challenge [31:25] ML platforms for internal tools [35:07] Impact of ML Education at Uber [39:30] Recommendations to companies who want to start an ML-Ed platform [41:12] Early ML Adoption Program [42:11] Homegrown or home-built platform [42:54] Feature creation to a course [45:24] ML Education at Uber: Frameworks Inspired by Engineering Principles [49:42] The Future of ML Education at Uber [52:28] Unclear ways to spread ML knowledge [54:20] Module for Generative AI and ChatGPT [55:05] Measurement of success [56:39] Wrap up

MLOps Coffee Sessions #155 with Matei Zaharia, The Birth and Growth of Spark: An Open Source Success Story, co-hosted by Vishnu Rachakonda. // Abstract We dive deep into the creation of Spark, with the creator himself - Matei Zaharia Chief technologist at Databricks. This episode also explores the development of Databricks' other open source home run ML Flow and the concept of "lake house ML". As a special treat Matei talked to us about the details of the "DSP" (Demonstrate Search Predict) project, which aims to enable building applications by combining LLMs and other text-returning systems. // About the guest: Matei has the unique advantage of being able to see different perspectives, having worked in both academia and the industry. He listens carefully to people's challenges and excitement about ML and uses this to come up with new ideas. As a member of Databricks, Matei also has the advantage of applying ML to Databricks' own internal practices. He is constantly asking the question "What's a better way to do this?" // Bio Matei Zaharia is an Associate Professor of Computer Science at Stanford and Chief Technologist at Databricks. He started the Apache Spark project during his Ph.D. at UC Berkeley, and co-developed other widely used open-source projects, including MLflow and Delta Lake, at Databricks. At Stanford, he works on distributed systems, NLP, and information retrieval, building programming models that can combine language models and external services to perform complex tasks. Matei’s research work was recognized through the 2014 ACM Doctoral Dissertation Award for the best Ph.D. dissertation in computer science, an NSF CAREER Award, and the US Presidential Early Career Award for Scientists and Engineers (PECASE). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://cs.stanford.edu/~matei/ https://spark.apache.org/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Matei on LinkedIn: https://www.linkedin.com/in/mateizaharia/ Timestamps: [00:00] Matei's preferred coffee [01:45] Takeaways [05:50] Please subscribe to our newsletters, join our Slack, and subscribe to our podcast channels! [06:52] Getting to know Matei as a person [09:10] Spark [14:18] Open and freewheeling cross-pollination [16:35] Actual formation of Spark [20:05] Spark and MLFlow Similarities and Differences [24:24] Concepts in MLFlow [27:34] DJ Khalid of the ML world [30:58] Data Lakehouse [33:35] Stanford's unique culture of the Computer Science Department [36:06] Starting a company [39:30] Unique advice to grad students [41:51] Open source project [44:35] LLMs in the New Revolution [47:57] Type of company to start with [49:56] Emergence of Corporate Research Labs [53:50] LLMs size context [54:44] Companies to respect [57:28] Wrap up

MLOps Coffee Sessions #154 with Waleed Kadous, ML Scalability Challenges, co-hosted by Abi Aryan. // Abstract Dr. Waleed Kadous, Head of Engineering at Anyscale, discusses the challenges of scalability in machine learning and his company's efforts to solve them. The discussion covers the need for large-scale computing power, the importance of attention-based models, and the tension between big and small data. // Bio Dr. Waleed Kadous leads engineering at Anyscale, the company behind the open-source project Ray, the popular scalable AI platform. Prior to Anyscale, Waleed worked at Uber, where he led overall system architecture, evangelized machine learning, and led the Location and Maps teams. He previously worked at Google, where he founded the Android Location and Sensing team, responsible for the "blue dot" as well as ML algorithms underlying products like Google Fit. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: anyscale.com https://www.youtube.com/watch?v=hzW0AKKqew4https://www.anyscale.com/blog/WaleedKadous-why-im-joining-anyscale Ray Summit: https://raysummit.anyscale.com/ Anyscale careers: https://www.anyscale.com/careersLearning Ray O'Reilly book. It's free to anyone interested. https://www.anyscale.com/asset/book-learning-ray-oreilly --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Waleed on LinkedIn: https://www.linkedin.com/in/waleedkadous/ Timestamps: [00:00] Waleed's preferred coffee [00:38] Takeaways [07:37] Waleed's background [13:16] Nvidia investment with Rey [14:00] Deep Learning use cases [17:52] Infrastructure challenges [22:01] MLOps level of maturity [26:42] Scale overloading [29:21] Large Language Models [32:40] Balance between fine-tuning forces prompts engineering [35:51] Deep Learning movement [42:05] Open-source models have enough resources [44:11] Ray [47:59] Value add for any scale from Ray [48:55] "Big data is dead" reconciliation [52:43] Causality in Deep Learning [55:16] AI-assisted Apps [57:59] Ray Summit is coming up in September! [58:49] Anyscale is hiring! [59:25] Wrap up

This exclusive podcast episode covers the key findings from the LLM in-production survey that we conducted over the past month.

For all the data to explore yourself use this link https://docs.google.com/spreadsheets/d/13wdBwkX8vZrYKuvF4h2egPh0LYSn2GQSwUaLV4GUNaU/edit?usp=sharing

Sign up for our LLM in-production conference happening on April 13th (TODAY) here:
https://home.mlops.community/home/events/llms-in-production-conference-2023-04-13

MLOps Coffee Sessions #153 with Rodolfo Núñez, Multilingual Programming and a Project Structure to Enable It, co-hosted by Abi Aryan. // Abstract It's really easy to mix different programming languages inside the same project and use a project template that enables easy collaboration. It's not about what language is better, but rather what language solves the given section of your problem better for you. // Bio Rodo has been working in the "Data Space" for almost 7 years. He was a Senior Data Scientist at Entel (a Chilean telecommunications company) and is now a Senior Machine Learning Engineer at the same company, where I also lead three mini teams dedicated to internal cybersecurity; design/promote continuous training for the entire Analytics team and also the whole company; and ensure the improvement of programming practices and code cleanliness standards. Rodo is currently in charge of helping the team put models into production and define the tools that we will use for it. He specializes in R, but he's language/tool agnostic: you should use the tool that best solves your current problem. Rodo studied Mathematical Engineering and MSc in Applied Mathematics at the University of Chile in addition to General Engineering at the École Centrale Marseille. Rodo really likes to share knowledge (bi-directionally) in whatever he thinks he can contribute. Some things that Rodo like teaching are Data Science, Math, Latin Dances, and whatever he thinks he can give to people. Rodo's other interests are computer games (especially Vermintide and Darktide), board games, and dancing to Latin rhythms. Also, he streams some games and Data Science related topics on Twitch. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.twitch.tv/en_codershttps://www.youtube.com/@en_codershttps://www.twitch.tv/rodonunezhttps://github.com/rodo-nunezhttps://github.com/en-coders-cl --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Rodo on LinkedIn: https://www.linkedin.com/in/rodonunez/ Timestamps: [00:00] Rodo's preferred coffee [00:16] Project structure [00:34] Introduction to Rodolfo Núñez [01:20] Takeaways [04:34] Check out our Meetups, podcasts, newsletters, TikTok, and blog posts! [05:50] Why data scientists should know how to code and code properly [10:32] Becoming a team player [14:02] Cookie cutter project [17:50] Markdown and Quarter over Jupyter notebooks [23:18] Data scientists' templates [30:06] Significance of scripts [33:30] Monolith to Microservices [34:33] Reproducibility [36:37] Entire event processing scripts [40:44] In-House cataloging solution [42:08] Data flows [46:00] Bonus topics! [47:23] Elbow methodology [50:17] Idea behind cross sampling [50:51] Machine Learning and MLOps Security at Entel [58:04] Wrap up

Worlds are colliding! This week we join forces with the hosts of the Practical AI podcast to discuss all things machine learning operations. We talk about how the recent explosion of foundation models and generative models is influencing the world of MLOps, and we discuss related tooling, workflows, perceptions, etc.

--------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

MLOps Coffee Sessions #152 with Keith Trnka, How A Manager Became a Believer in DevOps for Machine Learning. // Abstract Keith Trnka, a seasoned leader in the technology industry, set foot on the MLOps Podcast in a special episode where he shared insights into his experience leading data teams and machine learning teams, becoming a better software engineer, and overseeing a successful migration from a monolith to microservices in the healthcare sector without any downtime. Keith's background includes directing data science at 98.6, improving language models at swipe and nuance, and completing a Ph.D. thesis in language modeling for assistive technology. His work in these areas has contributed to the development of technology applications for healthcare, including telemedicine visits using natural language processing and machine learning. // Bio Keith has been in the industry for about 11 years. Most recently he was the Director of Data Science at 98point6 where we made telemedicine visits easier for doctors using natural language processing, machine learning, backend engineering, AWS, and frontend engineering. Prior to that, Keith improved the language models used in mobile phone keyboards at Swype and Nuance. And before that, He did his Ph.D. thesis in language modeling for assistive technology. Currently, Keith is traveling, mentoring, and doing a side project on machine translation. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Keith on LinkedIn: https://www.linkedin.com/in/keith-trnka/

MLOps Coffee Sessions #151 with Jean-Michel Daignan, ML in Production: A DS from Ubisoft Perspective, co-hosted by Abi Aryan. // Abstract As a data scientist himself, Jean-Michel has a unique perspective on the needs of data scientists when it comes to platform development. He talks about the non-invasive approach his team is taking to bring people onto the platform and their SDK, Merlin. The team is focused on tying machine learning products back to business use cases and the ROI they provide. Abby and Jean-Michel also discuss the use of generative AI and the importance of balancing delivering value and building things quickly. Jean-Michel's blog posts on the topic are recommended for further reading. // Bio The author of the blog "the-odd-dataguy.com" has been a data scientist for over 4.5 years at Ubisoft. Prior to joining the video game industry, Jean-Michel had a background in engineering from France and had previously worked in the energy sector. The blog focuses on topics related to data and machine learning, showcasing the author's expertise in the field. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Blog page: https://www.the-odd-dataguy.com/ Bringing Machine Learning to Production at Ubisoft (PydataMTL June22): https://www.the-odd-dataguy.com/2022/12/29/recap_pydata_mtl_june22/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Jean-Michel on LinkedIn: https://www.linkedin.com/in/jeanmicheldaignan/ Timestamps: [00:00] Jean-Michel's preferred beverage [00:19] Jean-Michel Daignan's background [00:28] Takeaways [04:30] Rate us and share the podcasts with your friends! [05:37] Jean-Michel's projects at Ubisoft [07:48] Jean-Michel's success as a Data Scientist [09:45] Ubisoft basics [10:40] Jean-Michel's success from the downfalls of being a data scientist [12:18] Building for data scientists' considerations [13:57] Differences in designing for data scientists in general [16:35] End twin pipelines and their functions [19:35] Major problems doing maintenance [20:53] Data quality ownership [22:33] Monitoring levels [24:25] Locomotive systems [26:14] Merlin [29:12] DS storage systems [31:09] Feature stores batch or streaming? [32:19] Bringing Machine Learning to Production at Ubisoft blog post [35:10] Features and recommendation systems [37:03] Playing games [38:21] Play data = play personalities [39:42] Deep learning in all the diffusion models or the foundation models [43:06] Servicing data scientists' needs [45:28] Ubisoft's data volume [48:00] Wrap up

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian. // Abstract Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event! // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links LLM in Production survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewform Virtual LLMs in Production Conference registration: https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13 Chinchilla papers: https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/ Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/ Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/ Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/ Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/ Timestamps: [00:00] Round table success to Virtual LLM in Production Conference on April 13th! [00:18] Register for the Virtual LLM in Production Conference now! [00:44] LLM in Production survey [01:40] Lightning round of introduction of speakers [04:34] Large Language Models definition [09:17] What do we consider large? [10:35] Thought process in use cases production [14:30] LLM open source huge movements [16:50] Problems qualifications [19:25] Production use cases frameworks directions [25:25] Open-source language models tokenizer [26:25] Language models democratization [29:25] Three categories for LLMs in Production [31:22] Latency at 2 levels [33:27] Defining production [34:57] Hitting the latency problems [38:20] Fundamental latency barrier [40:39] Latency use case requirement [44:25] Costs and the use cases [48:12] Product management involvement in costing [49:38] LLMs Hallucination definition [52:05] Building deterministic systems trust [55:21] Wrap up

MLOps Coffee Sessions #150 with Saahil Jain, The Future of Search in the Era of Large Language Models, co-hosted by David Aponte.

// Abstract Saahil shares insights into the You.com search engine approach, which includes a focus on a user-friendly interface, third-party apps, and the combination of natural language processing and traditional information retrieval techniques. Saahil highlights the importance of product thinking and the trade-offs between relevance, throughput, and latency when working with large language models.

Saahil also discusses the intersection of traditional information retrieval and generative models and the trade-offs in the type of outputs they produce. He suggests occupying users' attention during long wait times and the importance of considering how users engage with websites beyond just performance.

// Bio Saahil Jain is an engineer at You.com. At You.com, Saahil builds searching and ranking systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. His research work has been published in machine learning conferences such as EMNLP, NeurIPS Datasets & Benchmarks, and ACM-CHIL among others. He has publicly released various machine learning models, methods, and datasets, which have been used by researchers in both academic institutions and hospitals across the world, as part of an open-source movement to democratize AI research in medicine. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science at Columbia University and Stanford University respectively. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: http://saahiljain.me/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Saahil on LinkedIn: https://www.linkedin.com/in/saahiljain/ Timestamps [00:00] Saahil's preferred coffee [04:32] Saahil Jain's background [04:44] Takeaways [07:49] Search Landscape [12:57] Use cases exploration [14:51] Differentiating what to give to users [17:19] Search key challenges [20:05] Search objective relevance [23:22] MLOps Search and Recommender Systems [26:54] Addressing Latency Issues [29:41] Throughput presenting results [32:20] Compute challenges [34:24] Working at a small start-up [36:10] Citations critics [39:17] Use cases to build [40:40] Integrating to Leveraging You.com [42:26] Open AI [46:13] Interfacing with bugs [49:16] Staying focused [52:05] Retrieval augmented models [52:32] Closing thoughts [53:47] Wrap up

MLOps Coffee Sessions #149 with Jason McCampbell, The Challenges of Deploying (many!) ML Models, co-hosted by Abi Aryan and sponsored by Wallaroo.

// Abstract
In order to scale the number of models a team can manage, we need to automate the most common 90% of deployments to allow ops folks to focus on the challenging 10% and automate the monitoring of running models to reduce the per-model effort for data scientists. The challenging 10% of deployments will often be "edge" cases, whether CDN-style cloud-edge, local servers, or running on connected devices.

// Bio
Jason McCampbell is the Director of Architecture at Wallaroo.ai and has over 20 years of experience designing and building high-performance and distributed systems. From semiconductor design to simulation, a common thread is that the tools have to be fast, use resources efficiently, and "just work" as critical business applications.   

At Wallaroo, Jason is focused on solving the challenges of deploying AI models at scale, both in the data center and at "the edge". He has a degree in computer engineering as well as an MBA and is an alum of multiple early-stage ventures. Living in Austin, Jason enjoys spending time with his wife and two kids and cycling through the Hill Country.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: https://wallaroo.ai
MLOps at the Edge Slack channel: https://mlops-community.slack.com/archives/C02G1BHMJRL

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Jason on LinkedIn: https://www.linkedin.com/in/jasonmccampbell/

Timestamps:
[00:00] Jason's preferred coffee
[01:22] Takeaways
[06:06] MLOps at the Edge Slack channel
[06:36] Shoutout to Wallaroo!
[07:34] Jason's background
[09:54] Combining Edge and ML
[11:03] Defining Edge Computing
[13:21] Data transport restrictions
[15:02] Protecting IP in Edge Computing
[17:48] Decentralized Teams and Knowledge Sharing
[20:45] Real-time Data Analysis for Improved Store Security and Efficiency
[24:49] How to Ensure Statistical Integrity in Federated Networks
[26:50] Architecting ML at the Edge
[30:44] Machine Learning models adversarial attacks
[33:25] Pros and cons of baking models into containers
[34:52] Jason's work at Wallaroo
[38:22] Navigating the Market Edge
[40:49] Last challenges to overcome
[44:15] Data Science Use Cases in Logistics
[46:27] Vector trade-offs
[49:34] AI at the Edge challenges
[50:40] Finding the Sweet Spot
[53:46] Driving revenue
[55:33] Wrap up

MLOps Coffee Sessions #148 with Karl Fezer, Intelligence & MLOps co-hosted by Abi Aryan.

// Abstract
This conversation explores various topics including biases, defining intelligence, and the future of large language models and MLOps. Karl discusses his paper on defining intelligence and how it relates to the increasing interest in Artificial Intelligence. Karl shares his thoughts on the overlap between foundational models and MLOps, emphasizing the importance of making high-impact tasks more efficient and easier. The conversation touched on philosophical tangents but ultimately circled back to practical applications of these concepts.

// Bio
Karl graduated in 2014 from the University of Georgia with a Masters in Science in Artificial Intelligence. Since then, he has continued to stay on top of the latest iterations of Machine Learning and loves trying new open-source frameworks.   

For the last 6 years, he has been purely focused on AI Developer Relations. First at Mycroft, the voice assistant startup, then Intel, Arm, briefly at Wallaroo.ai, and now at Lockheed Martin.   

He currently lives in Seattle and spends his free time writing, reading, sailing, and camping.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Intelligence Primer paper by Karl Fezer and Andrew Sloss: https://arxiv.org/abs/2008.07324
Computing Machinery and Intelligence paper by Alan Turing: https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Karl on LinkedIn: https://www.linkedin.com/in/karlfezer/

MLOps Coffee Sessions #147 with Alex DeBrie, Something About Databases co-hosted by Abi Aryan.

// Abstract
For databases, it feels like we're in the middle of a big shift. The first 10-15 years of the cloud were mostly about using the same core infrastructure patterns but in the cloud (SQL Server, MySQL, Postgres, Redis, Elasticsearch).  

In the last few years, we're finally seeing data infrastructure that is truly built for the cloud. Elastic, scalable, resilient, managed, etc. Early examples were Snowflake + DynamoDB. The most recent ones are all the 'NewSQL' contenders (Cockroach, Yugabyte, Spanner) or the 'serverless' ones (Neon, Planetscale). Also seeing improvements in caching, search, etc. Exciting times!

// Bio
Alex is an AWS Data Hero and self-employed AWS consultant and trainer. He is the author of The DynamoDB Book, a comprehensive guide to data modeling with DynamoDB. Previously, he worked for Stedi and for Serverless, Inc., creators of the Serverless Framework. He loves being involved in the AWS & serverless community, and he lives in Omaha, NE with his family.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related
Links Key Takeaways from the DynamoDB Paper: https://www.alexdebrie.com/posts/dynamodb-paper/
Understanding Eventual Consistency in DynamoDB: https://www.alexdebrie.com/posts/dynamodb-eventual-consistency/
Two Scoops of Django 1.11: Best Practices for the Django Web Framework: https://www.amazon.com/Two-Scoops-Django-1-11-Practices/dp/0692915729CAP or no CAP?
Understanding when the CAP theorem applies and what it means: https://www.alexdebrie.com/posts/when-does-cap-theorem-apply/
Stop fighting your database/ The DynamoDB book: https://dynamodbbook.com/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Alex on LinkedIn: https://www.linkedin.com/in/alex-debrie/

Timestamps:
[00:00] Alex's preferred coffee
[00:27] Introduction to Alex DeBrie and DynamoDB
[01:05] Takeaways
[03:47] Please write down your reviews and you might have a book of Alex!
[04:57] Alex's journey from being an Attorney to becoming a Data Engineer
[07:31] Why engineering?
[10:07] Serverless Data
[12:54] Before Airflow
[15:41] Batch vs streaming
[17:03] Difficulties in Batch and Streaming side
[19:45] Modern data infrastructure databases
[24:37] Cloud Ecosystem Maturity
[27:59] New generation type of Snowflake
[29:47] Comparing databases
[30:58] What's next on connectors from 2 perspectives?
[34:25] Management services at the MLOps level
[36:38] DynamoDB
[39:32] Why do you like DynamoDB?
[41:00] Data used in DynamoDB and size limits
[43:46] Comparison of tradeoffs between DynamoDB and Redis
[45:52] Preferred opinionated databases
[48:43] CAP or no CAP? Understanding when the CAP theorem applies and what it means
[52:10] The DynamoDB book
[56:17] Chapter you want to expand on the book
[57:43] Next book to write
[59:25] ChatGPT iterations
[1:01:59] Data modeling book wished to be written
[1:03:27] Wrap up

MLOps Coffee Sessions #146 with Shalabh Chaudri, The Ops in MLOps - Process and People co-hosted by Abi Aryan.

// Abstract
Shalabh talks through their newfound appreciation for the MLOps perspective from a customer success standpoint. Shalabh's emphasis on setting realistic expectations and ensuring the delivery of promised value adds is particularly valuable.    

Generally, this episode provides a unique and insightful perspective on MLOps from the lens of customer success.

// Bio
Shalabh has worked in the MLOps domain since 2020 at Algorithmia and Union AI. His experience spans startups and small and large public companies. He has 10+ years of experience in the design, delivery, adoption, and business value realization of B2B infrastructure and platform solutions.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://www.union.ai/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Shalabh on LinkedIn: https://www.linkedin.com/in/shalabhchaudhri/

Timestamps:
[00:00] Shalabh's preferred coffee
[01:18] Takeaways
[02:57] Huge shout out to Union AI!
[03:46] Reviews
[05:26] Shalab's journey
[07:00] The people and process of MLOps
[10:25] Accuracy measures and Multiple Stakeholders
[13:01] UnionAI's success where others fall short
[14:45] Legacy equipment
[17:06] Legacy tools versus open source
[19:27] Cataloging solution
[22:51] Stakeholders and maturity levels
[24:26] People and Process in MLOps
[29:00] Collaboration for Machine Learning
[31:08] Overcoming challenges
[34:17] AI and leadership decision-making
[35:33] Legacy Companies and AI
[39:39] Common pitfalls  
[42:24] Neglecting ROI
[46:25] Speaking to each level
[49:50] Being realistic
[51:29] Becoming a champion
[53:08] Transitioning to machine learning
[55:25] Customer's Skill and Success needed in ML  
[57:46] Different sizes of companies 

MLOps Coffee Sessions #145 with Sahil Khanna, Griffin, ML platform at Instacart co-hosted by Mike Del Balso.

// Abstract
The conversation revolves around the journey of Instacart in implementing machine learning, starting from batch processing to real-time processing. The speaker highlights the importance of real-time processing for businesses and the relevance of Instacart's journey to other machine learning teams.   

Sahil emphasizes the soft factors, such as staying customer-focused and the right approach, that contributed to the success of Instacart's machine learning implementation. We also recommend two blog posts by Sahil about Instacart's journey.

// Bio
Sahil is currently a machine learning engineer at Instacart, where they are building a centralized platform for the training, deployment, and management of diverse ML applications. Before Instacart, Sahil developed ML training and inference platforms at Etsy.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mike on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/
Connect with Sahil on LinkedIn: www.linkedin.com/in/sahil-khanna-umd

MLOps Coffee Sessions #144 with Matthew Dombrowski, Non-traditional Career Paths in MLOps co-hosted by Mihail Eric.

// Abstract
Let's explore the different aspects of ML and data roles and the variety of responsibilities each role entails! This conversation emphasizes the need for understanding the unique insights each role provides and the similarities in responsibilities and soft skills that are required across different roles.   

This episode also highlights the significance of stakeholder alignment in the context of working in big companies and the importance of navigating these complexities for a successful career in ML.

// Bio
Matt has performed a number of MLOps positions including Solutions Consultant, Solutions Architect, and Product Manager from startups to large organizations. In his current role, Matt builds tools to help social media influencers discover unique and exciting Amazon products to recommend to their audiences.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/
Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewdombrowski/

Timestamps:
[00:00] Matt's preferred coffee
[00:28] Mihail's new creation
[05:09] Introduction to Matthew Dombrowski
[06:02] Takeaways
[08:30] Pizza and coffee nerds
[10:54] Data careers
[13:35] Matt's progression through the ml sphere
[20:10] Dealing with machine learning
[23:20] Transition from deep technical implementer to PM role
[27:42] Data is a product
[29:30] From start-ups to big companies
[32:41] Ambiguity of ML
[36:17] Matt's daily routine
[40:23] Social media influencers
[42:07] Developer advocate
[44:00] Stakeholder alignment
[49:41] Non-traditional career paths military influence
[54:11] Good ways to recommend people to get into ML
[57:56] MLOps Meetups all over the world
[59:00] Wrap up

MLOps Coffee Sessions #143 with Jill Chase & Manmeet Gujral, Investing in the Next Generation of AI & ML.

// Abstract
Investors are currently focusing on developer tooling and the foundational AI model movement, as they have seen explosive growth in this area. This podcast explores the impact of foundational models on investment thesis and the future of machine learning operations.

The discussion also touches on the idea of generative AI and large language models, and their potential impact on MLOps in the next 10 years. Jill and Manmeet from Capital G share their insights on this topic.

// Bio
Jill Chase
Jill is an investor at CapitalG where she focuses on enterprise software, with an emphasis on data infrastructure and AI/ML.

Prior to joining CapitalG, Jill worked in senior startup operating roles, both as the CEO of a private equity-backed business and as the founder of a Y Combinator-backed startup.

Jill graduated magna cum laude from Williams College with a dual degree in Economics and Psychology and was captain of the women’s basketball team. She came out to the West Coast to earn an MBA from the Stanford Graduate School of Business, but she was born and raised in Boston where she had the opportunity to cheer on the most impressive era of professional sports a city has ever experienced (Go Patriots).

She lives in the Bay Area with her husband where they spend weekends doing as many outside activities as possible, such as pickleball, tennis, hiking, and running.

Manmeet Gujral
Manmeet is a member of the CapitalG investment team where he focuses on enterprise software, AI & ML, open source, and product-led-growth companies. Prior to joining CapitalG in 2021, Manmeet worked in product marketing and operations at Tecton. Before that, he worked as a consultant at Bain & Company in San Francisco where he specialized in the go-to-market strategy for technology companies and private equity investment diligence. Manmeet is originally from Albany, New York, and graduated from Dartmouth College with a dual degree in Computer Science and Economics. Manmeet is highly opinionated about pizza, an avid New York sports fan, and always willing to share his latest house or hip-hop playlists.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Jill on LinkedIn: https://www.linkedin.com/in/jill-greenberg-chase-53747538/
Connect with Manmeet on LinkedIn: https://www.linkedin.com/in/manmeet-gujral/

Timestamps:
[00:00] Manmeet and Jill's preferred coffee
[00:25] Takeaways
[01:31] CapitalG, Jill and Manmeet's Background
[05:12] Sideswiping MLOps by Foundational Models
[08:50] MLOps space and the market revenue  
[14:50] Foundational models B to C style
[20:37] Foundational models taking over
[23:00] Uninnovative sentiments
[27:50] 2 prototypes of companies
[31:51] Finding product market fit
[36:20] MLOps market growth changes
[40:30] Monster valuations
[41:43] The ones that got away
[44:07] Wrap up

MLOps Coffee Sessions #142 with Murtuza Shergadwala, Approaches to Fairness and XAI co-hosted by Abi Aryan. This episode is sponsored by Fiddler AI.

// Abstract
The field of Explainable Artificial Intelligence (XAI) is continuously evolving, with an increasing focus on providing model-centric explanations in a human-centric manner. However, better frameworks and training for users are needed to fully utilize the potential of XAI tools.  

Additionally, there is a discrepancy in the approach to fairness in XAI, with the industry approaching it from a regulatory standpoint, while academia is engaging in more discussion and research on the topic.

// Bio
Dr. Murtuza Shergadwala is a data scientist at Fiddler AI. His background is in human-machine interaction and design decision-making. He received his Ph.D. from Purdue University in Mechanical Engineering. Prior to Fiddler, he was a postdoc at the Games User Interaction and Intelligence Lab at UC Santa Cruz where he focused on using Bayesian approaches for modeling cognition and investigating the theory of mind. He’s super passionate about fairness in AI for underrepresented communities.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://murtuzashergadwala.wixsite.com/murtuza
https://www.fiddler.ai/blog/detecting-intersectional-unfairness-in-ai-part-1

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Murtuza on LinkedIn: https://www.linkedin.com/in/murtuza-shergadwala/

Timestamps:
[00:00] Moto's preferred coffee
[00:35] Introduction to Murtuza Shergadwala
[01:06] Takeaways
[04:30] Huge shout out to Fiddler AI for sponsoring this episode!
[05:00] Don't forget to like, comment, and subscribe. Give us a rating  
[06:10] Moto's background and transition to Human-centric AI
[10:52] Decision-making behaviors of engineering designers in design contests
[15:10] Gaining insights from data decisions
[18:00] Defining latent variables
[20:32] Designer's perspective on building systems
[23:14] XAI as a movement
[27:47] Selling regulations and bridging the gap
[32:18] Data integrity towards detecting outliers alerting and data drifts
[34:32] Dealing with alerts and alert fatigue
[37:31] Approaches and their limitations
[39:10] Alert-level systems
[42:19] Alerts putting into practice
[45:30] Creative alerts
[47:02] One solution fits all?
[50:08] Wrap up

MLOps Coffee Sessions #141 with Stephen Bailey, Airflow Sucks for MLOps co-hosted by Joe Reis.

// Abstract
Stephen discusses his experience working with data platforms, particularly the challenges of training and sharing knowledge among different stakeholders. This talk highlights the importance of having clear priorities and a sense of practicality and mentions the use of modular job design and data classification to make it easier for end users to understand which data to use.   

Stephen also mentions the importance of being able to move quickly and not getting bogged down in the quest for perfection. We recommend Stephen's blog post "Airflow's Problem" for further reading.

// Bio
Stephen has worked as a data scientist, analyst, manager, and engineer, and loves all the domains equally. He currently works at Whatnot, a collectibles marketplace that focuses on live shopping, and has previously worked in privacy tech at Immuta. He has his Ph.D. from Vanderbilt University in educational cognitive neuroscience, but it has yet to help him understand why his three children are so crazy.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Airflow's Problem blog post: https://stkbailey.substack.com/p/airflows-problem
Airflow's Problem and the reception it got on Hacker News: https://news.ycombinator.com/item?id=32317558

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Joe on LinkedIn: https://www.linkedin.com/in/josephreis/
Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stkbailey/

Timestamps:
[00:00] Stephen's preferred coffee
[00:19] Introduction to co-host Joe Reis
[01:40] Takeaways
[06:29] Subscribe to our newsletters!
[06:55] Shout out to our sponsor, Wallaroo!
[08:05] Whatnot
[10:47] Stephen's side hustle
[14:35] Stephen's work breakdown at Whatnot
[18:03] Fundamental tensions in the data world
[21:27] Initial questions to answer that you were on the right path
[24:06] Recommender systems
[28:15] Coordinating with ML teams
[29:43] Daxter
[31:38] Too advanced, more challenging
[34:37] Orchestration layer
[36:14] Decision criteria
[39:23] Human design aspect of Daxter
[40:53] Orchestration layer centralization and sharing knowledge with stakeholders
[46:18] Airflow's Problem and the reception it got on Hacker News
[51:00] Wrap up

MLOps Coffee Sessions #140 with Sakib Dadi, The Evolution of ML Infrastructure sponsored by Wallaroo.

// Abstract
The toolkit and infrastructure empowering machine learning practitioners are advancing as ML adoption accelerates. We'll go through the current landscape of ML tooling, startups, and new projects from an investor's perspective.

// Bio
Sakib is a vice president in the San Francisco office where he primarily focuses on early-stage investments in developer platforms, data infrastructure, and machine learning. He has been involved with Bessemer’s investments in Prefect, Coiled, Arcion, Periscope Data (acquired by Sisense), Okera, npm (acquired by GitHub), and LaunchDarkly. Before joining Bessemer, Sakib worked in product at Viagogo, an international marketplace for buying and selling tickets for live events.     

Sakib also worked in the technology investment banking group at Morgan Stanley and as an engineer at Innova Dynamics (acquired by TPK), a startup manufacturing flexible touchscreen displays.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
 
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Sakib on LinkedIn: https://www.linkedin.com/in/sakib-dadi-77938937/

Timestamps:
[00:00] Sakib's preferred coffee
[00:13] Introduction to Sakib Dadi
[01:33] Sakib's background
[02:40] Shout out to this episode's Sponsor, Wallaroo!
[04:17] ML investing
[05:57] Investing regrets
[08:06] Transformers are today what would be tomorrow?
[09:18] Company that you wish existed now
[10:23] Current thoughts on the MLOps market
[12:32] MLOps transition to Generative AI
[15:52] Mind maps
[17:03] Jasper
[22:14] Intersection  
[24:10] Differences in models in-house
[26:08] Orchestration space
[28:23] Nuances of Monitoring
[29:20] Demetrios' theory on Monitoring
[31:48] Non-funded Monitoring Companies
[34:29] Investment risks
[36:55] Orchestration markets
[39:38] MLOps market at a plateau
[42:14] Vertical problems, vertical solutions  
[45:45] Sakib starting a company
[50:10] Structuring deals
[51:50] Infrastructure tools
[53:26] Firing a founder  
[53:49] Parting ways with a founder
[57:07] Wrap up

MLOps Coffee Sessions #139 with Alex Ratner, Putting Foundation Models to Use for the Enterprise co-hosted by Abi Aryan sponsored by Snorkel AI.

// Abstract
Foundation models are rightfully being compared to other game-changing industrial advances like steam engines or electric motors. They’re core to the transition of AI from a bespoke, less predictable science to an industrialized, democratized practice. Before they can achieve this impact, however, we need to bridge the cost, quality, and control gaps.   

Snorkel Flow Foundation Model Suite is the fastest way for AI/ML teams to put foundation models to use. For some projects, this means fine-tuning a foundation model for production dramatically faster by creating programmatically labeling training data. For others, the optimal solution will be using Snorkel Flow’s distill, combine, and correct approach to extract the most relevant knowledge from foundation models and encode that value into the right-sized models for your use case.   

AI/ML teams can determine which Foundation Model Suite capabilities to use (and in what combination) to optimize for cost, quality, and control using Snorkel Flow’s integrated workflow for programmatic labeling, model training, and rapid-guided iteration.

// Bio
Alex Ratner is the Co-founder and CEO of Snorkel AI and an Assistant Professor of Computer Science at the University of Washington.   

Prior to Snorkel AI and UW, he completed his Ph.D. in CS advised by Christopher Ré at Stanford, where he started and led the Snorkel open-source project, and where his research focused on applying data management and statistical learning techniques to emerging machine learning workflows such as creating and managing training data and applying this to real-world problems in medicine, knowledge base construction, and more. Previously, he earned his A.B. in Physics from Harvard University.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: www.snorkel.ai
Huge “foundation models” are turbo-charging AI progress: https://www.economist.com/interactive/briefing/2022/06/11/huge-foundation-models-are-turbo-charging-ai-progress
Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming: https://arxiv.org/abs/2203.01382
The Principles of Data-Centric AI Development: https://snorkel.ai/principles-of-data-centric-ai-development/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Alex on LinkedIn: https://www.linkedin.com/in/alexander-ratner-038ba239/

Timestamps: [00:00] Alex's preferred coffee [01:20] Introduction to Alex Ratner [02:34] Takeaways [04:04] Huge shoutout to our Sponsor, Snorkel AI! [04:39] Comment, rate us, and share our podcasts with your friends!  [04:50] Transfer Learning / Active Learning [11:30] Labeling Heuristics paper on Nemo [18:14] Geocentric AI [21:48] Enterprise use cases on Foundational Models [32:45] Foundational Models into the different Google products [38:36] Progress in Foundational Models [43:55] AutoML Models Baseline Accuracy [44:40] Hosting Infrastructure Snorkel Float vs GCP [46:53] Chris Re's venture capital firm / incubator / machine [51:00] Wrap

MLOps Coffee Sessions #138 with Dattaraj Rao, Explainability in the MLOps Cycle co-hosted by Vishnu Rachakonda.

// Abstract
When it comes to Dattaraj's interest, you'll hear about his top 3 areas in Machine Learning. What he sees as up and coming, what he's investing his company's time into and where he invests his own time.

Learn more about rule-based systems, deploying rule-based systems , and how to incorporate systems into more systems. there is no difference between ML systems and deploying models. It's just that this machine learning model is much smarter than traditional rule based models.

// Bio
Dattaraj Jagdish Rao is the author of the book “Keras to Kubernetes: The Journey of a Machine Learning Model to Production”. Dattaraj leads the AI Research Lab at Persistent and is responsible for driving thought leadership in AI/ML across the company. He leads a team that explores state-of-the-art algorithms in Knowledge Graphs, NLU, Responsible AI, MLOps and demonstrates applicability in Healthcare, Banking, and Industrial domains. Earlier, he worked at General Electric (GE) for 19 years building Industrial IoT solutions for Predictive Maintenance, Digital Twins, and Machine Vision.

Dattaraj held several Technology Leadership roles at Global Research, GE Power, and Transportation (now part of Wabtec). He led the Innovation team out of Bangalore that incubated video track inspection from an idea into a commercial Product. Dattaraj has 11 patents in Machine Learning and Computer Vision areas.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Keras to Kubernetes: The Journey of a Machine Learning Model to Production book:
https://www.amazon.com/Keras-Kubernetes-Journey-Learning-Production/dp/1119564832
Responsible Data Science Research | Talk @ VLDB 2022| Dattaraj Rao
https://www.youtube.com/watch?v=5_19KvSiy8s
Operationalizing AI/ML: Journey of an ML Model to Production | Masterclass by Dattaraj Rao
https://www.youtube.com/watch?v=Zk3RiiG07Us
Dattaraj Rao presenting workshop on MLOps at VISUM 2021
https://www.youtube.com/watch?v=wonUvbMDTUA
Machine Learning Design Patterns book:
https://www.oreilly.com/library/view/machine-learning-design/9781098115777/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Dattaraj on LinkedIn: https://www.linkedin.com/in/dattarajrao/

MLOps Coffee Sessions #137 with Niklas Kühl, Machine Learning Operations — What is it and Why Do We Need It? co-hosted by Abi Aryan.

// Abstract
The final goal of all industrial machine learning (ML) projects is to develop ML products and rapidly bring them into production.   

However, it is highly challenging to automate and operationalize ML products and thus many ML endeavors fail to deliver on their expectations. The paradigm of Machine Learning Operations (MLOps) addresses this issue.

// Bio
NIKLAS KÜHL studied Industrial Engineering & Management at the Karlsruhe Institute of Technology (KIT) (Bachelor and Master). During his studies, he gained practical experience in IT by working at Porsche in both national and international roles. Niklas has been working on machine learning (ML) and artificial intelligence (AI) in different domains since 2014. In 2017, he gained his PhD (summa cum laude) in Information Systems with a focus on applied machine learning from KIT. In 2020, he joined IBM.

As of today, Niklas engages in two complementary roles: He is head of the Applied AI in Services Lab at the Karlsruhe Institute of Technology (KIT), and, furthermore, he works as a Managing Consultant for Data Science at IBM. In his academic and practical projects, he is working on conceptualizing, designing, and implementing AI in Systems with a focus on robust and fair AI as well as the effective collaboration between users and intelligent agents. Currently, he and his team are actively working on different ML & AI solutions within industrial services, sales forecasting, production lines or even creativity. Niklas is internationally collaborating with multiple institutions like the University of Texas and the MIT-IBM Watson AI Lab.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: niklas.xyz
MLOps Newsletters: https://airtable.com/shrx9X19pGTWa7U3Y
Machine Learning Operations (MLOps): Overview, Definition, and Architecture paper: https://arxiv.org/abs/2205.02302

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Niklas on LinkedIn: https://www.linkedin.com/in/niklaskuehl/

Timestamps:
[00:00] Niklas' preferred coffee
[00:43] Introduction to Niklas Kühl
[01:16] Takeaways
[02:05] Subscribe to our newsletters and give us a rating here!
[02:54] Niklas background
[05:09] Scraping twitter data
[06:58] EV's conclusions
[08:24] NLP usage on Twitter
[10:26] Consumer behavior production
[12:03] Management and Machine Learning Systems Communication
[14:00] Current hype around Machine Learning
[15:10] Budgeting ML Productions
[18:15] Machine Learning Operations (MLOps): Overview, Definition, and Architecture paper
[22:56] Niklas' MLOps definiton  
[25:55] Navigating the idea of MLOps
[30:34] Return of Investment endeavor
[33:58] Full stack data scientist
[37:39] Defining success for different kinds of data science projects [41:06] Fun fact about Niklas [44:35] Other things Niklas do [47:02] The world is your oyster [50:57] Niklas' day to day  life [52:48] One lecture Niklas can drop in [53:57] Foundational models [58:20] Wrap up

MLOps Coffee Sessions #136 with Andrew Dye, Systems Engineer Navigating the World of ML co-hosted by David Aponte.

// Abstract
We don't hear that much about working at a very low level on this podcast but they are still very valid. Andrew is able to give us his take on why and what you need to keep in mind when you are working at these low levels and why it is very important when you are a Machine Learning Engineer and how the two can play together nicely.

Most MLOps teams are formed using existing people and exitsing engineers. More often than not you have to blend these various disciplines and it works well when there's a common goal.

// Bio
Andrew is a software engineer at Union and contributor to Flyte, a production grade data and ML orchestration platform. Prior to that he was a tech lead for ML Infrastructure at Meta, where he focused on ML training reliability.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/
Connect with Andrew on LinkedIn: https://www.linkedin.com/in/andrewwdye

Timestamps:
[00:00] Andrew's preferred coffee
[03:30] Introduction to Andrew Dye
[03:33] Takeaways
[07:32] Huge shoutout to our sponsors UnionML and UnionAI!
[07:48] Andrew's background
[10:08] Andrew's learning curve
[11:10] Bridging the gap between firmware space and MLOps
[12:18] In connection with Pytorch team
[12:54] Things that should have learned sooner
[14:54] Type of scale Andrew works on
[17:42] Distributed training at Meta
[19:55] Managing the huge search space
[22:18] Execution patterns programs
[23:20] Non-ML engineers dealing with ML engineers having the same skill set
[26:44] Pace rapid change adoptation
[29:18] Consensus challenges
[32:26] Abstractions making sense now
[34:53] Comparing to others
[39:21] General principles in UnionAI tooling
[41:54] Seeing the future
[43:54] Inter-task checkpointing
[44:52] Combining functionality with use cases
[46:17] Wrap up

MLOps Coffee Sessions #135 with Sasha Ovsankin and Rupesh Gupta, Real-time Machine Learning: Features and Inference co-hosted by Skylar Payne.  

// Abstract
Moving from batch/offline Machine Learning to more interactive "near" real-time requires knowledge, team, planning, and effort. We discuss what it means to do real-time inference and near-real-time features when to do this move, what tools to use, and what steps to take.  

// Bio
Sasha Ovsankin Sasha is currently a Tech Lead of Machine Learning Model Serving infrastructure at LinkedIn, worked also on Feathr Feature Store, Real-Time Feature pipelines, designed metric platforms at LinkedIn and Uber, and was co-founder in two startups. Sasha is passionate about AI, Software Craftsmanship, improvisational music, and many more things.  

Rupesh Gupta
Rupesh is a Sr. Staff Engineer in the AI team at LinkedIn. He has 10 years of experience in search and recommender systems.  

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links  

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/
Connect with Sasha on LinkedIn: https://www.linkedin.com/in/sashao/
Connect with Rupesh on LinkedIn: https://www.linkedin.com/in/guptarupesh

Timestamps:
[00:00] Sasha's and Rupesh's preferred coffee
[01:30] Takeaways
[07:23] Changes in LinkedIn
[09:21] "Real-time" Machine Learning in LibnkedIn
[13:08] Value of Feedback
[14:24] Technical details behind getting the most recent information integrated into the models
[16:53] Embedding Vector Search action occurrence
[18:33] Meaning of "Real-time" Features and Inference
[20:23] Are "Real-time" Features always worth that effort and always helpful?
[23:22] Importance of model application
[25:26] Challenges in "Real-time" Features
[30:40] System design review on Pinterest
[36:13] Successes of real-time features
[38:31] Learnings to share
[45:52] Branching for Machine Learning
[48:44] Not so talked about discussion of "Real-time"
[51:09] Wrap up

MLOps Coffee Sessions #134 with Jeremy Thomas Jordan, Building Threat Detection Systems: An MLE's Perspective co-hosted by Vishnu Rachakonda.

// Abstract
There is a clear pattern that we have been seeing with some of these greats in MLOps. So many use writing as a forcing function to learn about where they have holes in their understanding of something.   

If you are not writing, this episode is important as to why writing is important for your own development. Jeremy goes into writing in depth as to how beneficial it is for him to write and for him to see that he doesn't understand something if he cannot re-articulate it in writing.

// Bio
Jeremy is a machine learning engineer currently working at Duo Security where he focuses on building ML infrastructure to operate threat detection systems at scale. He previously worked at Proofpoint, where he built models for phishing and malware detection.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links Website:
https://www.jeremyjordan.me/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Visnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Jeremy on Twitter: https://twitter.com/jeremyjordan

MLOps Coffee Sessions #133 {Podcast BTS} with Chip Huyen, Real-time Machine Learning with Chip Huyen co-hosted by Vishnu Rachakonda.

// Abstract
Forcing functions and how you can supercharge your learning by putting yourself into a situation where you know you either have a responsibility to others to learn or accountability on you so you have to learn.  

It's not that hard when you think about streaming machine learning. It's not that big of a mental barrier to cross. It is simple in theory but maybe it's more complicated in practice and that's exactly where Chip's perspective is.

// Bio
Chip Huyen is a co-founder of Claypot AI, a platform for real-time machine learning. Previously, she was with Snorkel AI and NVIDIA. She teaches CS 329S: Machine Learning Systems Design at Stanford. She’s the author of the book Designing Machine Learning Systems, an Amazon bestseller in AI.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Landing page: https://claypot.ai
Designing Machine Learning Systems book:
https://www.amazon.com/Designing-Machine-Learning-Systems-Production-Ready/dp/1098107969

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Chip on LinkedIn: https://www.linkedin.com/in/chiphuyen/

MLOps Coffee Sessions #132 {Podcast BTS} with Ian Schweer, What is Data / ML Like on League? co-hosted by Skylar Payne.  

// Abstract
If you're not an avid gamer yourself, you have never really seen how ML might be used in the gaming space. It's so interesting to see the things that are different like full stories of players' games from start to finish.  

// Bio
On the surface, Ian is an excellent developer who gets things done. Underneath, he is much more. Ian is a reliable and trustworthy teammate who demonstrates an exceptional ownership mentality.  

Here's a fair share of Ian's job history:
2014 - UCI (With Skylar!)
2015 - Adobe Primetime (SWE)
2017 - Adobe Product and Customer Analytics (SWE)
2019 - Doordash Data Infra (SWE) Current - Riot Games on League  

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links Landing page:
https://www.riotgames.com/en

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/
Connect with Ian on LinkedIn: https://www.linkedin.com/in/ianschweer/

Timestamps:
[00:00] Ian's preferred coffee
[02:10] Takeaways
[05:14] Please hit the like button and leave us a review. Please subscribe also!
[05:45] Engineering Community Mental Health Awareness
[07:33] Coping mechanism
[09:29] Increase in video game playing  
[11:20] Ian's career progression
[17:55] Lessons to apply in the Data space
[24:23] Challenges at Riot
[34:18] Real-time element
[39:09] Ian's day-to-day responsibilities
[43:13] Analysis vs. Production Code Quality
[48:11] Tools and techniques on the reality of writing production codes
[55:00] What would you change your career into?
[57:00] Ian's best practices advise
[58:28] Ian's favorite video game
[59:58] Wrap-up

MLOps Coffee Sessions #131 {Podcast BTS} with Ethan Rosenthal, Let's Continue Bundling into the Database co-hosted by Mike Del Balso.

// Abstract
The relationship between ML Engineers and Product Managers is something that we don't talk about enough. We've got to get this right. If we don't get this right, either you're not focusing on the business problems in the right way or the Product Managers are not going to understand the tech appropriately to help make the right decisions.

// Bio
Ethan works on the Conversations Team at Square leading a team of Artificial Intelligence Engineers. Ethan's team builds applied AI solutions for Square Messages, a messaging hub for Square merchants to communicate with their customers. Prior to Square, Ethan spent time as a freelance data science consultant building machine learning products for a range of companies, from pre-seed startups to Fortune 100 enterprises.   

Ethan got his start in data science working at two different e-commerce startups, Birchbox and Dia&Co. Before data science, Ethan was an actual scientist and got his Ph.D. in experimental physics from Columbia University.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://www.ethanrosenthal.com/
Relevant blog posts:  
https://www.ethanrosenthal.com/2022/05/10/database-bundling/
https://www.ethanrosenthal.com/2022/07/19/materialize-ml-monitoring/
https://www.ethanrosenthal.com/2022/01/18/autoretraining-is-easy/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mike on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/
Connect with Ethan on LinkedIn: https://www.linkedin.com/in/ethanrosenthal/

Timestamps:
[00:00] Ethan's preferred coffee
[00:10] Introduction to co-host Mike Del Balso
[00:43] Takeaways
[04:10] Sign up for our newsletter!
[04:47] Ethan's team
[06:49] Ethan's team improvement
[08:40] Product manager role at Square
[10:39] Large Language Models
[12:22] Big questions to figure out
[15:45] Cost of false-positive
[18:20] Company Vocabulary
[20:11] MLOps concerns and challenges around Large Language Models
[23:42] Data learning management
[27:36] Leveling trade-offs
[30:25] Ethan's Database Bundling blog
[34:32] Feature Stores
[38:24] Streaming databases
[41:57] Best of both worlds trade-off highlight
[43:51] Rosenthal data
[46:40] Ethan's freelancing
[47:46] Risk-reward trade-off
[49:17] Ethan as a professor
[51:14] Wrap up

MLOps Coffee Sessions #130 {Podcast BTS} with Andrew Yates, Adversarial MLOps on Other People's Money: Lessons Learned from Ad Tech co-hosted by Abi Aryan.

// Abstract
Design ML to be components in a larger system with stable interfaces is not tracible to monitor the entire stack as a black box. You need intermediate ground-truth signals. Ads are designed in this way.

You can go from profitable to non-profitable real quick with ads. This will determine whether your company is around a year or two. You play with money and sometimes you play a lot of it so make sure that it's correct.

// Bio
Andrew Yates formerly led ads ranking, auction, and marketplace engineering and research teams at Facebook and Pinterest. He specializes in designing billion-dollar content marketplaces that maximize long-term revenue while protecting both seller and user experiences. Andrew has published over a dozen patents in online advertising optimization.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Andrew on LinkedIn: https://www.linkedin.com/in/andrew-yates-0217a985/

MLOps Coffee Sessions #129 {Podcast BTS} with Catherin Breslin, Voice and Language Tech co-hosted by Adam Sroka.

// Abstract
Back in the day, Speech Recognition was its own thing. It's a very different flavor of Data Science. You could not use a lot of the tools. It wouldn't cross over to this type of machine learning.

Now, with the advancements, Speech Recognition and Machine learning are coming in together. It's interesting to hear right from someone with a Ph.D. level working with some of the biggest companies in the world doing it. The fact that something like Alexa is lots of models back to back and just fathom the complexity of that is quite cool!

// Bio
Catherine is a machine learning scientist and consultant based in Cambridge UK, and the founder of Kingfisher Labs consulting. Since completing her Ph.D. at the University of Cambridge in 2008, Catherine has commercial and academic experience in automatic speech recognition, natural language understanding, and human-computer dialogue systems, having previously worked at Cambridge University, Toshiba Research, Amazon Alexa, and Cobalt Speech. Catherine has been excited by the application of research to real-world problems involving speech and language at scale.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
www.catherinebreslin.co.uk
https://catherinebreslin.medium.com/
MLOps Community Newsletter: https://airtable.com/shrx9X19pGTWa7U3YTwitter: https://twitter.com/catherinebuk

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka
Connect with Catherine on LinkedIn: https://www.linkedin.com/in/catherine-breslin-0592423a/

Timestamps:
[00:00] Catherine's preferred coffee
[01:50] Takeaways
[03:59] Introduction to Catherine Breslin
[05:04] Subscribe to our newsletter!
[06:25] Catherine's background
[08:13] Speech Recognition trajectory
[09:36] Challenges around technologies and tools
[11:34] Reflective trend
[13:02] Developer experiences hiccups
[15:09] Speech Recognition use case backup
[16:56] Toshiba research
[17:48] Transition from a research lab to working in the industry
[20:01] Unit test of Speech Recognition
[20:56] Alexa
[22:33] Maturity process of Speech Recognition
[26:48] Speech Recognition unrecognizing challenges
[30:38] Mechanical Terk
[33:00] Social media listening
[34:05] Pipeline models and speed of Speech Recognition
[36:48] Development of Speech Recognition excited about
[37:23] Data from people for the Speech Recognition system vs Scowering news vs watching Youtube for a long time
[40:00] Disappearing Languages
[41:30] Future of an online practice partner
[43:17] Speech-to-speech translation
[44:04] Interesting ways to use unfamiliar models to achieve a result
[45:40] Meeting transcriptions
[48:37] First toy problem of a new Speech Recognition learner
[51:37] Kingfisher Labs' problems to tackle
[52:18] Off-the-shelf solution
[53:38] Translation layer
[54:15] Connect with Catherine on Twitter and LinkedIn for available jobs
[54:43] Wrap up

MLOps Coffee Sessions #128 with Simon Thompson, Managing Machine Learning Projects co-hosted by Abi Aryan.

// Abstract
It's a cliche to say that choosing and running the algorithms is only a small part of a typical ML project but despite that it's true! Setting up and organizing the project, dealing with the data asset, getting to the heart of the business problem, assessing and choosing the models, and integrating them with the business processes in production are all at least as time-consuming and important.   

Simon has written a book that talks about how these different activities need to be orchestrated and executed and he hopes that it might be useful for people who are starting out managing ML projects and help them avoid some of the crunches and catches that seem to trip people up.

// Bio
Simon has been building and running ML projects since 1994 (when he started his Ph.D. in MachineLearning). His first commercial project was for the Royal Navy, and since then he has worked in Telecom, Defense, Consultancy, Manufacturing, and Finance. This means Simon has experienced a wide range of working environments and different types of projects. As well as working in a variety of commercial environments Simon collaborated on EU research projects, UK Government funded research projects and worked as an industrial rep on three MIT consortia (BigData@CSAIL, Systems That Learn, and the CISR Data Research Board).

Simon was also an industrial fellow at the Alan Turing Institute for a year. This means that he has also seen a lot of the communities' practices and concerns as they developed, and he had the chance to put them into use in a commercial environment.   

Right now, Simon is working for a technology consultancy called GFT, and his job there is primarily to deliver ML projects for companies in the capital markets such as investment banks, although we also do work in retail banking, insurance, and manufacturing.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://medium.com/@sgt101
Managing Machine Learning Projects From design to deployment book by Simon Thompson:
https://www.manning.com/books/managing-machine-learning-projects
MLOps Community Newsletter: https://airtable.com/shrx9X19pGTWa7U3Y
Language processing. Simon Thompson CO545 Lecture 10: https://docplayer.net/211236676-Language-processing-simon-thompson-co545-lecture-10.html

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Simon on LinkedIn: https://www.linkedin.com/in/simon-thompson-025a7/

MLOps Coffee Sessions #127 with Niall Murphy & Todd Underwood, Reliable ML co-hosted by David Aponte.

// Abstract
By applying an SRE mindset to machine learning, authors and engineering professionals Cathy Chen, Kranti Parisa, Niall Richard Murphy, D. Sculley, Todd Underwood, and featured guest authors show you how to run an efficient and reliable ML system. Whether you want to increase revenue, optimize decision-making, solve problems, or understand and influence customer behavior, you'll learn how to perform day-to-day ML tasks while keeping the bigger picture in mind. (Book description from O'Reilly)

MLOps Coffee Sessions #127 with Niall Murphy & Todd Underwood, Reliable ML co-hosted by David Aponte.   // Abstract By applying an SRE mindset to machine learning, authors and engineering professionals Cathy Chen, Kranti Parisa, Niall Richard Murphy, D. Sculley, Todd Underwood, and featured guest authors show you how to run an efficient and reliable ML system. Whether you want to increase revenue, optimize decision-making, solve problems, or understand and influence customer behavior, you'll learn how to perform day-to-day ML tasks while keeping the bigger picture in mind. (Book description from O'Reilly)  It was great that they wrote this book in the first place in a space that's new and lots of people are entering it with a lot of questions and this book clarifies those questions. It was also great to have all of their experiences documented in this one book and there's a lot of value in putting them all in one place so that people can benefit from it.

// Bio
Niall Murphy
Niall has been interested in Internet infrastructure since the mid-1990s. He has worked with all of the major cloud providers from their Dublin, Ireland offices - most recently at Microsoft, where he was the global head of Azure Site Reliability Engineering (SRE). His books have sold approximately a quarter of a million copies worldwide, most notably the award-winning Site Reliability Engineering, and he is probably one of the few people in the world to hold degrees in Computer Science, Mathematics, and Poetry Studies. He lives in Dublin, Ireland, with his wife and two children.

Todd Underwood
Todd is a Director at Google and leads Machine Learning for Site Reliability Engineering Director. He is also the Site Lead for Google’s Pittsburgh office. ML SRE teams build and scale internal and external ML services and are critical to almost every Product Area at Google.    

Before working at Google, Todd held a variety of roles at Renesys.  He was in charge of operations, security, and peering for Renesys’s Internet intelligence services which are now part of Oracle's Cloud service. He also did product work for some early social products that Renesys worked on. Before that Todd was the Chief Technology Officer of Oso Grande, an independent Internet service provider (AS2901) in New Mexico.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Reliable Machine Learning book by Cathy Chen, Niall Richard Murphy, Kranti Parisa, D. Sculley, Todd Underwood: https://www.oreilly.com/library/view/reliable-machine-learning/9781098106218/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Niall on LinkedIn: https://www.linkedin.com/in/niallm/
Connect with Todd on LinkedIn: https://www.linkedin.com/in/toddunder/

MLOps Coffee Sessions #126 with George Mathew, ML Unicorn Start-up Investor Tells-IT-All.

// Abstract
What's so enticing about enterprise software? It's incredible to see George's idea and vision to invest in generationally enduring companies.  

Let's look at the way how George likes to structure deals with companies while he's reviewing them and let's look at the MLOps ecosystem through the eyes of the investors.

// Bio
George Mathew joins Insight Partners as a Managing Director focused on venture stage investments in AI, ML, Analytics, and Data companies as they are establishing product/market Fit.  

He brings 20+ years of experience developing high-growth technology startups including most recently being CEO of Kespry. Prior to Kespry, George was President & COO of Alteryx where he scaled the company through it’s IPO (AYX). Previously he held senior leadership positions at SAP and salesforce.com. He has driven company strategy, led product management and development, and built sales and marketing teams.    

George holds a Bachelor of Science in Neurobiology from Cornell University and a Masters in Business Administration from Duke University, where he was a Fuqua Scholar.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://www.insightpartners.com/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with George on LinkedIn: https://www.linkedin.com/in/gmathew/

MLOps Coffee Sessions #125 with Rafael Pierre, Deploying Real-time ML Models in Minutes with Databricks Model Serving V2 co-hosted by Ryan Russon.

// Abstract
From our experience helping customers in the Data and AI field, we learned that the most challenging part of Machine Learning is deploying it. Putting models into production is complex and requires additional pieces of infrastructure as well as specialized people to take care of it - this is especially true if we are talking about real-time REST APIs for serving ML models.  

With Databricks Model Serving V2, we introduce the idea of Serverless REST endpoints to the platform. This allows teams to easily deploy their ML models in a production-grade platform with a few mouse clicks (or lines of code 😀).

// Bio
Rafael has worked for 15 years in data-intensive fields within finance in multiple roles: software engineering, product management, data engineering, data science, and machine learning engineering.   

At Databricks, Rafael has fun bringing all these topics together as a Solutions Architect to help our customers become more and more data-driven.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://mlopshowto.com
Airflow Summit 2022:
https://youtu.be/JsYEOdRBgREING
Data Engineering Meetup:
https://www.youtube.com/watch?v=gJoxX1rRZJI
MLOps World Virtual Summit NYC 2022:
https://drive.google.com/file/d/1EXsqmLfrPAsV9i6h6pGfJxVjMO9y6u9a/view?usp=sharing

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Ryan on LinkedIn: https://www.linkedin.com/in/ryanrusson/
Connect with Rafael on LinkedIn: https://www.linkedin.com/in/rafaelpierre

Lightning Sessions #2 with Aparna Dhinakaran, Co-Founder and Chief Product Officer, and Jason Lopatecki, CEO and Co-Founder of Arize. Lightning Sessions is sponsored by Arize

// Abstract  
Monitoring embeddings on unstructured data is not an easy feat let's be honest. Most of us know what it is but don't understand it one hundred percent.  

Thanks to Aparna and Jason of Arize for breaking down embedding so clearly. At the end of this Lightning talk, we get to see a demo of how Arize deals with unstructured data and how you can use Arize to combat that.

// Bio
Aparna Dhinakaran
Aparna is the Co-Founder and Chief Product Officer at Arize AI, a pioneer, and early leader in machine learning (ML) observability. A frequent speaker at top conferences and thought leader in the space, Dhinakaran was recently named to the Forbes 30 Under 30. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she built several core ML Infrastructure platforms, including Michaelangelo.

Aparna has a BA from Berkeley's Electrical Engineering and Computer Science program, where she published research with Berkeley's AI Research group. She is on a leave of absence from the Computer Vision Ph.D. program at Cornell University.

Jason Lopatecki
Jason is the Co-founder and CEO of Arize AI, a machine learning observability company. He is a garage-to-IPO executive with an extensive background in building marketing-leading products and businesses that heavily leverage analytics. Prior to Arize, Jason was co-founder and chief innovation officer at TubeMogul where he scaled the business into a public company and eventual acquisition by Adobe.   

Jason has hands-on knowledge of big data architectures, programmatic advertising systems, distributed systems, and machine learning and data processing architectures. In his free time, Jason tinkers with personal machine learning projects as a hobby, with a special interest in unsupervised learning and deep neural networks. He holds an electrical engineering and computer science degree from UC Berkeley.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// Related Links
https://arize.com/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Aparna on LinkedIn: https://www.linkedin.com/in/aparnadhinakaran/
Connect with Jason on LinkedIn: https://www.linkedin.com/in/jason-lopatecki-9509941/

Timestamps:
[00:00] Introduction to the topic
[01:13] Troubleshooting unstructured ML models is difficult
[01:40] Challenges with monitoring unstructured data
[02:10] How data looks like
[03:02] Embeddings are the backbone of unstructured models
[03:28] ML teams need a common tool
[04:06] What are embeddings?
[05:08] The real WHY behind AI
[06:41] ML observability for unstructured data
[07:08] Index and Monitor every Embedding
[08:05] Measuring drift of unstructured data
[08:54] Interactive visualizations  
[09:34] Fix underlying data issue
[09:44] Data-centric AI workflow
[10:08] Demo of the product
[12:48] Wrap up

MLOps Coffee Sessions #124 with Kush Varshney, Distinguished Research Staff Member and Manager IBM Research, Trustworthy Machine Learning co-hosted by Krishnaram Kenthapadi.

// Abstract
Trustworthy ML is a way of thinking and something to be worked on and operationalized throughout the entire machine learning development lifecycle, starting from the problem specification phase that should include diverse stakeholders.

// Bio
Kush R. Varshney was born in Syracuse, New York in 1982. He received a B.S. degree (magna cum laude) in electrical and computer engineering with honors from Cornell University, Ithaca, New York, in 2004. He received the S.M. degree in 2006 and the Ph.D. degree in 2010, both in electrical engineering and computer science at the Massachusetts Institute of Technology (MIT), Cambridge. While at MIT, he was a National Science Foundation Graduate Research Fellow.

Dr. Varshney is a distinguished research staff member and manager with IBM Research at the Thomas J. Watson Research Center, Yorktown Heights, NY, where he leads the machine learning group in the Foundations of Trustworthy AI department. He was a visiting scientist at IBM Research - Africa, Nairobi, Kenya in 2019. He is the founding co-director of the IBM Science for Social Good initiative. He applies data science and predictive analytics to human capital management, healthcare, olfaction, computational creativity, public affairs, international development, and algorithmic fairness, which has led to recognitions such as the 2013 Gerstner Award for Client Excellence for contributions to the WellPoint team and the Extraordinary IBM Research Technical Accomplishment for contributions to workforce innovation and enterprise transformation. He conducts academic research on the theory and methods of trustworthy machine learning. His work has been recognized through best paper awards at the Fusion 2009, SOLI 2013, KDD 2014, and SDM 2015 conferences and the 2019 Computing Community Consortium / Schmidt Futures Computer Science for Social Good White Paper Competition. He self-published a book entitled 'Trustworthy Machine Learning in 2022, available at http://www.trustworthymachinelearning.com. He is a senior member of the IEEE.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links


--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Krishnaram on LinkedIn: https://www.linkedin.com/in/krishnaramkenthapadi
Connect with Kush on LinkedIn: https://www.linkedin.com/in/kushvarshney/

MLOps Coffee Sessions #123 with Gleb Abroskin, Machine Learning Engineer at Funcorp, RECOMMENDER SYSTEM: Why They Update Models 100 Times a Day co-hosted by Jake Noble.

// Abstract
FunCorp was a top 10 app store. It was a very popular app that has a ton of downloads and just memes. They need a recommendation system on top of that. Memes are super tricky because they're user-generated and they evolve very quickly. They're going to live and die by the Recommender System in that product.

It's incredible to see FunCorp's maturity. Gleb breaks down the feature store they created and the velocity they have to be able to create a whole new pipeline in a new model and put it into production after only a month!

// Bio
Gleb make models go brrrrr. He doesn't know what is expected in this field, to be honest, but Gleb has experience in deploying a lot of different ML models for CV, speech recognition, and RecSys in a variety of languages (C++, Python, Kotlin) serving millions of users worldwide.

/ MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Putting a two-layered recommendation system into production -
https://medium.com/@FunCorp/putting-a-two-layered-recommendation-system-into-production-b8caaf61393d
Practical Guide to Create a Two-Layered Recommendation System -
https://medium.com/@FunCorp/practical-guide-to-create-a-two-layered-recommendation-system-5486b42f9f63
Ten Mistakes to Avoid When Creating a Recommendation System -
https://medium.com/@FunCorp/ten-mistakes-to-avoid-when-creating-a-recommendation-system-8268ed60aeba
Applying Domain-Driven Design And Patterns: With Examples in C# and .net 1st Edition by Jimmy Nilsson:
https://www.amazon.com/Applying-Domain-Driven-Design-Patterns-Examples/dp/0321268202

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Jake on LinkedIn: https://www.linkedin.com/in/jakednoble/
Connect with Gleb on LinkedIn: https://www.linkedin.com/in/gasabr/

Timestamps:
[00:00] Introduction to Gleb Abroskin
[00:50] Takeaways
[05:39] Breakdown of FunCorp teams
[06:47] FunCorp's team ratio
[07:41] FunCorp team provisions
[08:48] Feature Store vision
[10:16] Matrix factorization
[11:51] Fairly modular fairly thin infrastructure
[12:26] Distinct models with the same feature
[13:08] FunCorp's definition of Feature Store
[15:10] Unified API
[15:55] FunCorp's scaling direction
[17:01] Level up as needed
[17:38] Future of FunCorp's Feature Store
[18:37] Monitoring investment in the space
[19:43] Latency for business metrics
[21:04] Velocity to production
[23:10] 30-day retention struggle
[24:45] Back-end business stability
[27:49] Recommender systems
[30:34] Back-end layer headaches
[32:04] Missing piece of the whole Feature Store picture
[33:54] Throwing ideas turn around time
[36:37] Decrease time to market
[37:41] Continuous training pipelines or produce an artifact
[39:33] Worst-case scenario
[40:38] Realistic estimation of a new model deployment
[41:42] Recommender Systems' future velocity  
[43:07] A/B Testing launch - no launch decision
[46:32] Lightning question
[47:08] Wrap up

MLOps Coffee Sessions #122 with Hannes Hapke, Machine Learning Engineer at Digits Financial, Inc., Scaling Similarity Learning at Digits co-hosted by Vishnu Rachakonda.

// Abstract
Machine Learning in a product is a double-edged sword. It can make a product more useful but it depends on assumed and strictly defined behavior from users.  

Hannes walks through the entirety of their machine learning pipeline, how they implemented it, what the elements are, what the learning looks like, and what tooling looks like.   

Hannes maps out what good data hygiene looks like not only from the machine learning perspective down to the software engineering, design, and backend engineering, all the way to the data engineering perspectives.

// Bio
Hannes was the first ML engineer at Digits, where he built the MLOPs foundation for their ML team. His interest in production machine learning ranges from building ML pipelines to scaling similarity-based ML to process millions of banking transactions daily.   

Prior to Digits, Hannes implemented ML solutions for a number of applications, incl. retail, health care, or ERP companies.
He co-author two machine learning books:
* Building Machine Learning Pipeline (O'Reilly)
* NLP in Action (Manning)

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links


--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/

Timestamps:
[00:00] Introduction to Hannes Hapke
[01:37] Takeaways
[02:40] Design supercharges machine learning
[05:48] Building Machine Learning Pipeline book
[08:09] Updating the edition
[09:37] Abstract away
[11:52] Approach of crossover
[16:04] Training serving skew
[20:42] Tools using continuous integration and deployment
[25:25] Human in the loop touch point
[27:44] Data backfilling update
[30:06] Work and Products of Digits
[32:26] Digit Boost
[35:30] The first machine learning engineer
[39:55] Structured data in good shape, good data processing perspective, concept-educated teams  
[43:33] Digits is hiring!
[43:55] Machine Learning struggles
[47:10] Design decision
[49:49] Data or machine learning literacy
[51:30] Data Hygiene
[52:49] Rapid fire questions
[54:47] Wrap up

MLOps Coffee Sessions #121 with Luis Ceze, CEO and Co-founder of OctoML, Bringing DevOps Agility to ML co-hosted by Mihail Eric.  

// Abstract
There's something about this idea where people see a future where you don't need to think about infrastructure. You should just be able to do what you do and infrastructure happens.  

People understand that there is a lot of complexity underneath the hood and most data scientists or machine learning engineers start deploying things and shouldn't have to worry about the most efficient way of doing this.

// Bio
Luis Ceze is Co-Founder and CEO of OctoML, which enables businesses to seamlessly deploy ML models to production making the most out of the hardware. OctoML is backed by Tiger Global, Addition, Amplify Partners, and Madrona Venture Group. Ceze is the Lazowska Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he has taught for 15 years.

Luis co-directs the Systems and Architectures for Machine Learning lab (sampl.ai), which co-authored Apache TVM, a leading open-source ML stack for performance and portability that is used in widely deployed AI applications.   

Luis is also co-director of the Molecular Information Systems Lab (misl.bio), which led pioneering research in the intersection of computing and biology for IT applications such as DNA data storage. His research has been featured prominently in the media including New York Times, Popular Science, MIT Technology Review, and the Wall Street Journal. Ceze is a Venture Partner at Madrona Venture Group and leads their technical advisory board.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Landing page: https://octoml.ai/
The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics by Daniel James Brown:
https://www.amazon.com/Boys-Boat-Americans-Berlin-Olympics/dp/0143125478

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/
Connect with Luis on LinkedIn: https://www.linkedin.com/in/luis-ceze-50b2314/

Timestamps:
[00:00] Introduction to Luis Ceze
[06:28] MLOps does not exist
[10:41] Semantics argument
[16:25] Parallel programming standpoint
[18:09] TVM
[22:51] Optimizations
[24:18] TVM in the ecosystem
[27:10] OctoML's further step
[30:42] Value chain
[33:58] Mature players
[35:48] Talking to SRE's and Machine Learning Engineers
[36:32] Building OctoML
[40:20] My Octopus Teacher
[42:15] Environmental effects of Sustainable Machine Learning
[44:50] Bridging the gap from OctoML to biological mechanisms
[50:02] Programmability
[57:13] Academia making the impact
[59:40] Rapid fire questions
[1:03:39] Wrap up

MLOps Coffee Sessions #120 with David Stein, Senior Staff Software Engineer at LinkedIn, Feathr: LinkedIn's Enterprise-Grade, High-Performance Feature Store co-hosted by Skylar Payne.

// Abstract
When David started building Feathr, Feature Stores did not exist. That was not a term floating around at all. This was definitely one of the OG Feature Stores for sure!  

We hear how the LinkedIn team got to this point, having an open source release, and how they used LinkedIn as an incubator to build a great product.

// Bio
David Stein is a tech lead at LinkedIn working on machine learning feature infrastructure. He is the original architect of Feathr and continues to contribute to its development as well as to other parts of LinkedIn's machine learning platform.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://engineering.linkedin.com/blog/2022/open-sourcing-feathr---linkedin-s-feature-store-for-productive-m
* https://github.com/linkedin/feathr
* https://engineering.linkedin.com/blog/2022/open-sourcing-feathr---linkedin-s-feature-store-for-productive-m
* https://azure.microsoft.com/en-us/blog/feathr-linkedin-s-feature-store-is-now-available-on-azure/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/
Connect with David on LinkedIn: https://www.linkedin.com/in/steindavidj/

Timestamps:
[00:00] Introduction to David Stein
[00:30] Takeaways
[06:58] David and Skylar's reunion
[08:34] Feathr's background
[12:04] Lessons learned in building Feathr
[16:35] Scale of Feathr
[20:25] Systems interaction
[24:15] Standardization
[29:20] Importance and better difference of Feathr
[34:30] Feature Stores' evolution and more generally MLOps
[37:57] Challenges in real-time
[40:09] Going real-time, you're not ready!
[42:24] Use cases strong for real-time unleveraged now
[45:50] Proud about
[47:58] Going back in time
[50:47] Feathr as a separate company?
[52:53] LinkedIn is hiring!

MLOps Coffee Sessions #119 with Hien Luu, Sr. Engineering Manager of DoorDash, MLOps at DoorDash: 3 Principles for Building an ML Platform That Will Sustain Hypergrowth co-hosted by Skylar Payne.

// Abstract
Machine Learning plays a big part at DoorDash in terms of what they do on a daily basis. It powers many of their core infrastructures.  

When it comes to DoorDash's business, they have to be leveraging machine learning and it is such a huge piece of the business that it is critical.

// Bio
Hien Luu is an Engineering Manager at DoorDash, leading the Machine Learning platform team at DoorDash. He is particularly passionate about the intersection between Artificial Intelligence and Big Data. He is the author of the Beginning Apache Spark 3 book.  He has given presentations at various conferences like Data+AI Summit, MLOps World, Deep Learning Summit, and apply() conference.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
engineering.linkedin.com/hadoop/user-engagement-powered-apache-pig-and-hadoop
* https://doordash.engineering/2020/07/20/enabling-efficient-machine-learning-model-serving/
* https://doordash.engineering/2020/11/19/building-a-gigascale-ml-feature-store-with-redis/
* https://doordash.engineering/2021/03/04/building-a-declarative-real-time-feature-engineering-framework/
* https://doordash.engineering/2021/05/20/monitor-machine-learning-model-drift/
* https://doordash.engineering/2021/01/26/computational-graph-machine-learning-ensemble-model-support/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/
Connect with Hien on LinkedIn: https://www.linkedin.com/in/hienluu/

Timestamps:
[00:00] Introduction of DoorDash team
[01:58] Overview of DoorDash
[03:32] DoorDash's platform
[13:23] Experimenting and testing new models
[15:15] Experience transferring
[17:16] Effective engagement with customers
[24:15] Team sizes
[25:37] Metrics
[33:25] App for users
[34:04] Using Databricks and Snowflake together
[37:49] Supporting power users
[40:17] Advice and experiences
[43:53] Wrap up

MLOps Coffee Sessions #118 with Olalekan Elesin, Director of Data Platform & Data Architect at HRS Product Solutions GmbH, co-hosted by Vishnu Rachkonda.

// Abstract
You don't have infinite resources? Call out your main metrics! Focus on the most impactful things that you could do for your data scientists. Olalekan joined us to talk about his experience previously building a machine learning platform at Scaleout24.   

From our standpoint, this is the best demonstration and explanation of the role of technical product management in ML that we have on the podcast so far!

// Bio
Olalekan Elesin is a technologist with a successful track record of delivering data-driven technology solutions that leverages analytics, machine learning, and artificial intelligence. He combines experience working across 2 continents and 5 different market segments ranging from telecommunications, e-commerce, online marketplaces, and current business travel.   

Olalekan built the AI Platform 1.0 at Scout24 and currently leads multiple data teams at HRS Group. He is an AWS Machine Learning Community Hero in his spare time.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
What Customers Want: Using Outcome-Driven Innovation to Create Breakthrough Products and Services book by Anthony Ulwick: https://www.amazon.com/What-Customers-Want-Outcome-Driven-Breakthrough/dp/0071408673
Empowered: Ordinary People, Extraordinary Products by Marty Cagan:  
https://www.amazon.com/EMPOWERED-Ordinary-Extraordinary-Products-Silicon/dp/111969129X
How to Avoid a Climate Disaster: The Solutions We Have and the Breakthroughs We Need by Bill Gates:
https://www.amazon.com/How-Avoid-Climate-Disaster-Breakthroughs/dp/059321577X

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Olalekan on LinkedIn: https://www.linkedin.com/in/elesinolalekan/

Timestamps:
[00:00] Introduction to Olalekan Elesin
[00:42] Takeaways
[02:52] Situation at Scaleout24
[07:53] Data landscape engineer and architect
[11:27] Depiction of events
[13:53] Platform approach investment
[15:59] Exceptional need or opportunity to the most intense need
[17:41] Long-tail pieces
[22:01] Metrics
[24:15] Nitty-gritty product works
[26:00] Educating people metrics
[30:02] Upskilling fundamentals of the product discipline
[34:05] Investing in AWS
[37:53] Best-of-breed tools
[44:34] Continuous development for AutoML
[47:26] Rapid fire questions
[52:19] Wrap up

MLOps Coffee Sessions #117 with Chad Sanderson, Head of Product, Data Platform at Convoy, Data Engineering for ML co-hosted by Josh Wills.

// Abstract
Data modeling is building relationships between core concepts within your data. The physical data model shows how the relationships manifest in your data environment but then there's the semantic data model, the way that entity relationship design is extracted away from any data-centric implementation.  

Let's do the good old fun of talking about why data modeling is so important!

// Bio
Chad Sanderson is the Product Lead for Convoy's Data Platform team, which includes the data warehouse, streaming, BI & visualization, experimentation, machine learning, and data discovery.

Chad has built everything from feature stores, experimentation platforms, metrics layers, streaming platforms, analytics tools, data discovery systems, and workflow development platforms. He’s implemented open source, SaaS products (early and late-stage) and has built cutting-edge technology from the ground up. Chad loves the data space, and if you're interested in chatting about it with him, don't hesitate to reach out.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://odsc.com/speakers/scaling-machine-learning-with-data-mesh/ https://docs.google.com/presentation/d/1rVtltHkRkP_JaGZdkAS3U_SXfr5Gg-RP980FKXh0YNU/edit?usp=sharing
Josh Wills will be teaching a course on Data Engineering for Machine Learning in September here:
https://www.getsphere.com/ml-engineering/data-engineering-for-machine-learning

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Josh on LinkedIn: https://www.linkedin.com/in/josh-wills-13882b/
Connect with Chad on LinkedIn: https://www.linkedin.com/in/chad-sanderson/

Timestamps:
[00:00] Introduction of the new co-host Josh Wills  
[00:54] Introduction to Chad Sanderson
[01:46] Josh will lead a course for Machine Learning in mid-September
[02:16] Data modeling blog post of Chad
[06:10] Idea of Strategy
[09:40] Modern cloud data warehouses  
[17:01] Layering on contracts
[20:38] Scaling at larger companies
[25:30] Carrot-stick strategy
[34:27] Second and third-order effects
[39:53] Stockholm Syndrome
[41:22] Quality checks at Slack
[45:28] Success in two main ways according to Chad
[47:35] Completely and utterly different universes
[53:42] Product use case to push semantic events
[56:00] Pattern of analysis of the sequence of events
[57:23] Wrap up

MLOps Coffee Sessions #116 with Shawn Kyzer, Principal Data Engineer at Thoughtworks (Spain), Scaling Machine Learning with Data Mesh co-hosted by Adam Sroka.

// Abstract
You can't just get something done by using tools. You need to go much deeper than that and it is very clear that Data Mesh is the same thing. You have to educate the organization about the movement.  

In this session, Shawn broke down the cultural piece of data mesh and how many parallels there are with the MLOps Movement when it comes to the cultural side of MLOps.

// Bio
Shawn is passionate about harnessing the power of data strategy, engineering, and analytics in order to help businesses uncover new opportunities. As an innovative technologist with over 13 years of experience, Shawn removes technology as a barrier and broadens the art of the possible for business and product leaders. His holistic view of technology and emphasis on developing and motivating strong engineering talent, with a focus on delivering outcomes whilst minimising outputs, is one of the characteristics which sets him apart from the crowd.

Shawn’s deep technical knowledge includes distributed computing, cloud architecture, data science, machine learning, and engineering analytics platforms. He has years of experience working as a consultant practitioner for a variety of prestigious clients ranging from secret clearance level government organizations to Fortune 500 companies.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://odsc.com/speakers/scaling-machine-learning-with-data-mesh/ https://docs.google.com/presentation/d/1rVtltHkRkP_JaGZdkAS3U_SXfr5Gg-RP980FKXh0YNU/edit?usp=sharing

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/
Connect with Shawn on LinkedIn: https://www.linkedin.com/in/shawn-kyzer-msit-mba-b5b8a4b/

Timestamps:
[00:00] Introduction to Shawn Kyzer
[00:43] Takeaways
[04:00] Data Mesh for ML projects
[11:22] The signal for the exploratory part of a new modeling project
[14:13] Ownership and centralization
[16:20] Lack of technology and some implementations literature
[17:10] Python stronghold from Microsoft blogs
[23:09] Integration with self-serve data platform
[25:31] Starting a platform team
[30:04] Quick wins
[32:09] Metrics monitoring
[34:18] Metrics break up
[38:32] Limit to capabilities and not worth doing
[41:39] Culture and technology holds
[44:03] Setting the foundation
[46:53] Unforeseen benefits
[52:19] Lightning question

MLOps Coffee Sessions #115 with Flaviu Vadan, Senior Software Engineer at Dyno Therapeutics, How Hera is an Enabler of MLOps Integrations co-hosted by Vishnu Rachakonda.

// Abstract
Flaviu talks about the internal ML platform at Dyno Therapeutics called Hera. His team uses Hera as an internal innovation engine to help discover new breakthroughs with machine learning in the biotech healthcare industry.

/ Bio
Flaviu is a Senior Software Engineer at Dyno Therapeutics, the leading organization in the design of novel gene therapy vectors with transformative delivery properties for a vast landscape of human diseases. Flaviu comes from a background focused on Bioinformatics, which is a field that combines Computer Science, Mathematics, and Biology. He took stints in academia by working as a research assistant in Computer Science and Bioinformatics labs before joining Dyno Therapeutics to work on machine-guided design of adeno-associated viruses (AAVs).   

At Dyno, Flaviu works on compute and core infrastructure, DevOps, MLOps, and approaches that combine AI/ML to design AAVs in silico. He is also the author and maintainer of Hera, a Python SDK that facilitates access to Argo Workflows by making workflow construction and submission easy and accessible.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Flaviu on LinkedIn: https://www.linkedin.com/in/flaviuvadan/

Timestamps:
[00:00] Introduction to Flaviu Vadan
[00:50] Takeaways
[02:06] Share this episode with a friend!
[03:20] What Dyno does
[05:44] CRISPR and Gene Editing
[06:21] Kidney transplants and using pig organs
[07:31] Deciding what genes to put in the body  
[07:48] Role of ML at Dyno
[10:07] Higher dose
[13:41] Process of Machine Learning Deployment and Productionizing at Dyno
[16:22] Proliferation of models
[17:31] Building the internal platform
[19:37] Interaction with data, translation to compute layer, evaluation
[24:21] Venn diagram for MLOps
[27:06] Leveraging Argo Workflows
[30:34] Hera
[35:28] Open sourcing
[38:44] Human power at Dyno
[41:17] Wrap up

MLOps Coffee Sessions #114 with Marc Lindner, Co-Founder COO and Amr Mashlah, Head of Data Science of eezylife Inc., Product Enrichment and Recommender Systems co-hosted by Skylar Payne.

// Abstract
The difficulties of making multi-modal recommender systems. How it can be easy to know something about a user but very hard to know the same thing about a product and vice versa? For example, you can clearly know that a user wants an intellectual movie, but it is hard to accurately classify a movie as intellectual and fully automated.

// Bio
Marc Lindner Marc has a background in Knowledge Engineering. He's Always extremely product-focused with anything to do with Machine Learning.   

Marc built several products working together with companies such as Lithium Technologies etc. and then co-Founded eezy.

Amr Mashlah
Amr is the head of data science at eezy, where he leads the development of their recommender engine. Amr has a master's degree in AI and has been working with startups for 6 years now.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Children of Time book by Adrian Tchaikovsky:  
https://www.amazon.com/Children-Time-Adrian-Tchaikovsky/dp/0316452505

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/
Connect with Marc on LinkedIn: https://www.linkedin.com/in/marc-lindner-883a0883/
Connect with Amr on LinkedIn: https://www.linkedin.com/in/mashlah/

MLOps Coffee Sessions #113 with Leanne Fitzpatrick, Director of Data Science of Financial Times, Building Better Data Teams co-hosted by Mihail Eric.

// Abstract
We spent a lot of time talking about data tooling but we maybe spent not as much time talking about data organizations and efficiently running and organizing data teams.   

What about starting with limitations instead of aspirations? Right constraints instead of the north star? In this session, let's learn more about a realistic take on the state of data organizations of today.

// Bio
Leanne is Director of Data Science at the Financial Times and is a passionate data leader with experience building and developing empowered data science and analytics teams in a variety of businesses. Leanne is in her element when developing and implementing strategic, technical, and cultural solutions to getting machine learning and data science into the operational ecosystem.

Leanne is an active part of the data and technology community, sharing innovation and insights to encourage best practices, from Manchester, UK to Austin, TX, and is an Advisory Panel Board Member. Outside of all things data you can ask Leanne about her golf swing (it’s not good - yet), her passion for American Football (specifically the Cincinnati Bengals), her latest sewing project, and her love for good music, food, and whisky.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related
Links Children of Time book by Adrian Tchaikovsky:  
https://www.amazon.com/Children-Time-Adrian-Tchaikovsky/dp/0316452505

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/
Connect with Leanne on LinkedIn: https://www.linkedin.com/in/leanne-kim-fitzpatrick-29204341/

Timestamps:
[00:00] Introduction to Leanne Fitzpatrick
[04:23] Write us your suggestions!
[05:43] Tri-pawed dog called Seaweed!
[08:43] How to architect data teams
[14:44] Organizational deficiencies
[19:19] Tensions and conflicts for starters
[24:07] Misunderstandings from marketing
[25:59] The Middle Layer
[28:48] Data science work at publications
[31:11] Mystique of going to real-time
[35:29] Third parties with fraud
[37:40] Augmenting data practitioners with third-party tools
[41:00] Principle of reinventing the wheel and avoiding undifferentiated heavy lifting  
[46:29] Different Abstraction Layer recommendations
[48:42] RN Production
[51:56] Will Python eats RN Production away?
[56:05] Julia as a dark horse
[56:39] Future of RN Production
[58:00] Rapid fire questions

MLOps Coffee Sessions #112 with Xiangrui Meng, Principal Software Engineer of Databricks, MLX: Opinionated ML Pipelines in MLflow co-hosted by Vishnu Rachakonda.

// Abstract
MLX is to enable data scientists to stay mostly within their comfort zone utilizing their expert knowledge while following the best practices in ML development and delivering production-ready ML projects, with little help from production engineers and DevOps.

// Bio
Xiangrui Meng is a Principal Software Engineer at Databricks and an Apache Spark PMC member. His main interests center around simplifying the end-to-end user experience of building machine learning applications, from algorithms to platforms and to operations.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Good Strategy Bad Strategy: The Difference and Why It Matters book by Richard Rumelt:
https://www.amazon.com/Good-Strategy-Bad-Difference-Matters/dp/0307886239

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Xiangrui on LinkedIn: https://www.linkedin.com/in/mengxr/


Timestamps:
[00:00] Introduction to Xiangrui Meng
[00:39] Takeaways
[02:09] Xiangrui's background
[03:38] What kept Xiangrui in Databricks
[07:33] What needs to be done to get there
[09:20] Machine Learning passion of Xiangrui
[11:52] Changes in building that keep you fresh for the future
[14:35] Evolution core challenges to real-time and use cases in real-time
[17:33] DevOps + DataOps + ModelOps = MLOps
[19:21] MLFlow Support
[21:37] Notebooks to production debates  
[25:42] Companies tackling Notebooks to production
[27:40] MLOoops stories
[31:03] Opinionated MLOps productionizing in a good way
[40:23] Xiangrui's MLOps Vision
[44:47] Lightning round
[48:45] Wrap up

MLOps Coffee Sessions #111 with Samuel Partee, Principal Applied AI Engineer of Redis, More than a Cache: Turning Redis into a Composable, ML Data Platform co-hosted by Mihail Eric. This episode is sponsored by Redis.

// Abstract
Pushing forward the Redis platform to be more than just the web-serving cache that we've known it up to now. It seems like a natural progression for the platform, we see how they're evolving to be this AI-focused, AI native serving platform that does vector similarity, feature stored provides those kinds of functionalities.

// Bio
A Principal Applied AI Engineer at Redis, Sam helps guide the development and direction of Redis as an online feature store and vector database.   

Sam's background is in high-performance computing including ML-related topics such as distributed training, hyperparameter optimization, and scalable inference.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
https://partee.io
Redis VSS demo: https://github.com/Spartee/redis-vector-search
Redis Stack: https://redis.io/docs/stack/
Github - https://github.com/Spartee  
OSS org Sam co-founded at HPE/Cray - https://github.com/CrayLabs
This paper last year was some of the best research and collaborations Sam has been a part of. The Paper is published here: https://www.sciencedirect.com/science/article/pii/S1877750322001065?via%3Dihub
Do you really need an extra database for vectors? https://databricks.com/dataaisummit/session/emerging-data-architectures-approaches-real-time-ai-using-redis
Blink: The Power of Thinking Without Thinking by Malcolm Gladwell,  Barry Fox,  Irina Henegar (Translator): https://www.goodreads.com/book/show/40102.Blink

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/
Connect with Sam on LinkedIn: www.linkedin.com/in/sam-partee-b04a1710a

Timestamps:
[00:00] Introduction to Samuel Partee
[00:24] Takeaways
[02:46] Updates on the Community
[05:17] Start of Redis
[08:10] Vision for Vector Search
[11:05] Changing the narrative going from the "Cache" for all servers and web endpoints
[14:35] Clear value prop on demos
[20:17] Vector Database
[26:26] Features with benefits
[28:41] AWS Spend
[30:39] Vector Database upsell model and bureaucratic convenience  
[32:08] Distributed training hyperparameter optimization and scalable inference
[35:03] Core infrastructural advancement
[36:55] Tools movement to help
[39:00] Using Machine Learning at scale in numerical simulations with SmartSim: An application to ocean climate modeling (published paper) [42:52] Future applications of tech to get excited with
[44:20] Lightning round
[47:48] Wrap up

MLOps Coffee Sessions #110 with David Bayliss, Chief Data Scientist of LexisNexis Risk Solutions, Just Fetch the Data and then... co-hosted by Vishnu Rachakonda.

// Abstract
Composing data to extract features can be a significant problem. Key factors are the data size, compliance restrictions, and real-time data. Ethics (and law) can drive extremely complex audit requirements. In the cloud, you can do anything - at a price.

// Bio
One of the creators of the world's first big data platform (HPCC);  David has been tackling big data problems for two decades. A mathematician, compiler writer, and data sponge with more than five dozen patents spanning platforms linking, and search.

Most inventors think outside the box; David can't even remember where the box is. He leads the team that creates their core Data Science methods used by hundreds of data scientists.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Interesting insight in this post. Would be cool to learn from David about his view on things
https://www.google.com/url?q=https://www.linkedin.com/posts/david-bayliss-426556a_datascience-platform-portability-activity-6913448643303759872-2dqq?utm_source%3Dlinkedin_share%26utm_medium%3Dmember_desktop_web&sa=D&source=calendar&ust=1649078059106132&usg=AOvVaw26wAevExeEfW_AdZSA8UhF

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with David on LinkedIn: https://www.linkedin.com/in/david-bayliss-426556a/

Timestamps:
[00:00] Introduction to David Bayliss
[01:03] Takeaways
[04:56] LexisNexis and David's role
[07:15] Evolution of LexisNexis in 20 years with so many use cases
[08:51] Role of David in structuring data for working with data change
[14:32] Data management and data access
[17:45] Unique challenges of scale, use case, and diversity at LexisNexis
[24:47] Tardis Iron Box
[30:05] Iron Box translation
[32:56] JVM for data science
[34:24] Iron Box meaning
[36:52] Metadata with PII
[39:08] Detrimental privacy / Hairy Kneecap Theory
[40:57] Speeding things up and Anonymized linking
[46:47] What kept David working at LexisNexis?
[50:30] Wrap up