Foundation Model Series: Advancing Endoscopy with Matt Schwartz from Virgo

What if a routine endoscopy could do more than just detect disease by actually predicting treatment outcomes and revolutionizing precision medicine? In this episode of Impact AI, Matt Schwartz, CEO and Co-Founder of endoscopy video management and AI analysis platform Virgo, discusses how AI and machine learning are transforming endoscopy.

Tuning in, you’ll learn how Virgo’s foundation model, EndoDINO, trained on the largest endoscopic video dataset in the world, is unlocking new possibilities in gastroenterology. Matt also shares how automated video capture, AI-powered diagnostics, and predictive analytics are reshaping patient care, with a particular focus on improving treatment for inflammatory bowel disease (IBD). Join us to discover how domain-specific foundation models are redefining healthcare and what this means for the future of precision medicine!

Key Points:

• An introduction to Matt Schwartz and Virgo’s mission.
• The importance of video documentation in endoscopy and its impact on healthcare.
• Machine learning’s role in automating endoscopic video capture and clinical trial recruitment.
• Building the EndoDINO foundation model to unlock endoscopy data for precision medicine.
• Data collection: the process of gathering 130,000+ procedure videos for model training.
• Foundation model development using self-supervised learning and DINOv2.
• Model development challenges, from hyper-parameter tuning to domain-specific adjustments.
• Applying EndoDINO to predict inflammatory bowel disease (IBD) treatment responses.
• Commercializing EndoDINO through licensing to health systems and pharma companies.
• The future of foundation models in endoscopy: expanding applications beyond GI diseases.
• Advice for AI startup founders to prioritize data capture as a foundation for AI success.
• Insight into Virgo’s vision to transform IBD treatment and preventative care.

Quotes:

“There's a massive amount of endoscopic video data being generated across a wide range of endoscopic procedures, and nobody was capturing that data – [Virgo] realized early on that endoscopy data could hold the key to unlocking all sorts of opportunities in precision medicine.” — Matt Schwartz

“With the foundation model paradigm, you can compress a lot of heavy compute needs into a single model and then build different applications on top of the foundation. This is going to have a positive impact on the clinical deployment of foundation models.” — Matt Schwartz

“Our foundation model can turn something like a routine colonoscopy into a precision medicine screening tool for IBD patients.” — Matt Schwartz

“There are a lot of untapped data resources in healthcare. If a founder can build a first product that is the data capture engine, it will set them up for a ton of future success when it comes to AI development.” — Matt Schwartz

Links:

Virgo

Matt Schwartz on LinkedIn

Matt Schwartz on X

EndoML

Introducing EndoDINO: A Breakthrough in Endoscopic AI

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.