AI + a16z

In this episode of AI + a16z, Ambience cofounder and chief scientist Nikhil Buduma joins Derrick Harris to discuss the nuances of using AI models to build vertical applications (including in his space, health care), and why industry acumen is at least as important as technical expertise. Nikhil also shares his experience of having a first-row seat to key advances in AI — including the transformer architecture — which not only allowed his company to be an early adopter, but also gave him insight into the types of problems that AI could solve in the future.

Here's an excerpt of Nikhil explaining the importance of understanding your buyer:

"If you believe that the most valuable companies are going to fall out of some level of vertical integration between the app layer and the model layer, [that] this next generation of incredibly valuable companies is going to be built by founders who've spent years just obsessively becoming experts in an industry, I would recommend that someone actually know how to map out the most valuable use cases and have a clear story for how those use cases have synergistic, compounding value when you solve those problems increasingly in concert together. 

"I think the founding team is going to have to have the right ML chops to actually build out the right live learning loops, build out the ML ops loops to measure and to close the gap on model quality for those use cases. [But] the model is actually just one part of solving the problem. 

"You actually need to be thoughtful about the product, the design, the delivery competencies to make sure that what you build is integrated with the right sources of the enterprise data that fits into the right workflows in the right way. And you're going to have to invest heavily in the change management to make sure that customers realize the full value of what they're buying from you. That's all actually way more important than people realize."

Learn more:

Fundamentals of Deep Learning

Follow everyone on X:

Nikhil Buduma

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of AI + a16z, a16z General Partner Jennifer Li joins MotherDuck Cofounder and CEO Jordan Tigani to discuss DuckDB's spiking popularity as the era of big data wanes, as well as the applicability of SQL-based systems for AI workloads and the prospect of text-to-SQL for analyzing data.

Here's an excerpt of Jordan discussing an early win when it comes to applying generative AI to data analysis:

"Everybody forgets syntax for various SQL calls. And it's just like  in coding. So there's some people that memorize . . . all of the code base, and so they don't need auto-complete. They don't need any copilot. . . . They don't need an ID; they can just type in Notepad. But for the rest of us, I think these tools are super useful. And I think we have seen that these tools have already changed how people are interacting with their data, how they're writing their SQL queries.

"One of the things that we've done . . .  is we focused on improving the experience of writing queries. Something we found is actually really useful is when somebody runs a query and there's an error, we basically feed the line of the error into GPT 4 and ask it to fix it. And it turns out to be really good. 

". . . It's a great way of letting you stay in the flow of writing your queries and having true interactivity."

Learn more:

Small Data SF conference

DuckDB

Follow everyone on X:

Jordan Tigani

Jennifer Li

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of the AI + a16z podcast, Black Forest Labs founders Robin Rombach, Andreas Blattmann, and Patrick Esser sit down with a16z general partner Anjney Midha to discuss their journey from PhD researchers to Stability AI, and now to launching their own company building state-of-the-art image and video models. They also delve into the topic of openness in AI, explaining the benefits of releasing open models and sharing research findings with the field.

Learn more:

Flux

Keep the code to AI open, say two entrepreneurs

Follow everyone on X:

Robin Rombach

Andreas Blattmann

Patrick Esser

Anjney Midha

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode, a16z General Partner Vijay Pande walks us through the past two decades of applying software engineering to the life sciences — from the Folding@Home project that he launched, through AlphaFold and more. He also discusses the major opportunities for AI to transform medicine and health care, as well as some pitfalls that founders in that space need to watch out for.

Here's an excerpt of Vijay discussing how AlphaFold and other projects revolutionized biology research not just because of their algorithms, but because of how they introduced software engineering into the field:

"I think the key thing about AlphaFold that really got people excited was not just the AI part, because people have been using machine learning. And so that part was there. I think it was how fast, at least to me, an engineering approach could make a big jump in this field. Because this was a field largely addressed by academics, and academics would have a lab of maybe 20 [or] 30 people — some of the bigger ones, maybe slightly bigger. And of that, these are graduate students working on their PhDs. It's very different than having a team of professional programmers and engineers going after the problem. 

"And so that jump in team ability, plus the technology, I think was very critical for the jump in results. And also, finally, I think having a company like Google say, 'You know, this is a problem we're excited about and we're interested in,' and that AI and biology is something that is an area of great interest to them . . . was a huge flag to plant."

Learn more:

a16z Bio + Health

Folding@Home

AlphaFold

Raising Health podcast

Follow everyone on X:

Vijay Pande

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of the AI + a16z podcast, a16z General Partner Anjney Midha speaks with PromptFoo founder and CEO Ian Webster about the importance of red-teaming for AI safety and security, and how bringing those capabilities to more organizations will lead to safer, more predictable generative AI applications. They also delve into lessons they learned about this during their time together as early large language model adopters at Discord, and why attempts to regulate AI should focus on applications and use cases rather than models themselves.

Here's an excerpt of Ian laying out his take on AI governance:

"The reason why I think that the future of AI safety is open source is that I think there's been a lot of high-level discussion about what AI safety is, and some of the existential threats, and all of these scenarios. But what I'm really hoping to do is focus the conversation on the here and now. Like, what are the harms and the safety and security issues that we see in the wild right now with AI? And the reality is that there's a very large set of practical security considerations that we should be thinking about. 

"And the reason why I think that open source is really important here is because you have the large AI labs, which have the resources to employ specialized red teams and start to find these problems, but there are only, let's say, five big AI labs that are doing this. And the rest of us are left in the dark. So I think that it's not acceptable to just have safety in the domain of the foundation model labs, because I don't think that's an effective way to solve the real problems that we see today.

"So my stance here is that we really need open source solutions that are available to all developers and all companies and enterprises to identify and eliminate a lot of these real safety issues."

Learn more:

Securing the Black Box: OpenAI, Anthropic, and GDM Discuss

Security Founders Talk Shop About Generative AI

California's Senate Bill 1047: What You Need to Know

Follow everybody on X:

Ian Webster

Anjney Midha

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of the AI + a16z podcast, Command Zero cofounder and CTO Dean de Beer joins a16z's Joel de la Garza and Derrick Harris to discuss the benefits of training large language models on security data, as well as the myriad factors product teams need to consider when building on LLMs. 

Here's an excerpt of Dean discussing the challenges and concerns around scaling up LLMs:

"Scaling out infrastructure has a lot of limitations: the APIs you're using, tokens, inbound and outbound, the cost associated with that — the nuances of the models, if you will. And not all models are created equal, and they oftentimes are very good for specific use cases and they might not be appropriate for your use case, which is why we tend to use a lot of different models for our use cases . . .

"So your use cases will heavily determine the models that you're going to use. Very quickly, you'll find that you'll be spending more time on the adjacent technologies or infrastructure. So, memory management for models. How do you go beyond the context window for a model? How do you maintain the context of the data, when given back to the model? How do you do entity extraction so that the model understands that there are certain entities that it needs to prioritize when looking at new data? How do you leverage semantic search as something to augment the capabilities of the model and the data that you're ingesting? 

"That's where we have found that we spend a lot more of our time today than on the models themselves. We have found a good combination of models that run our use cases; we augment them with those adjacent technologies."

Learn more:

The Cuckoo's Egg

1995 Citigroup hack

Follow everyone on social media:

Dean de Beer

Joel de la Garza

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of the AI + a16z podcast, Anyscale cofounder and CEO Robert Nishihara joins a16z's Jennifer Li and Derrick Harris to discuss the challenges of training and running AI models at scale; how a focus on video models — and the huge amount of data involved — will change generative AI models and infrastructure; and the unique experience of launching a company out of the UC-Berkeley Sky Computing Lab (the successor to RISElab and AMPLab).

Here's a sample of the discussion, where Robert explains how generative AI has turbocharged the appetite for AI capabilities within enterprise customers:

"Two years ago, we would talk to companies, prospective customers, and AI just wasn't a priority. It certainly wasn't a company-level priority in the way that it is today. And generative AI is the reason a lot of companies now reach out to us . . . because they know that succeeding with AI is essential for their businesses, it's essential for their competitive advantage.

"And time to market matters for them. They don't want to spend a year hiring an AI infrastructure team, building up a 20-person team to build all of the internal infrastructure, just to be able to start to use generative AI. That's something they want to do today."

At another point in the discussion, he notes on this same topic:

"One dimension where we try to go really deep is on the developer experience and just enabling developers to be more productive. This is a complaint we hear all the time with machine learning teams or infrastructure teams: They'll say that they hired all these machine learning people, but then the machine learning people are spending all of their time managing clusters or working on the infrastructure. Or they'll say that it takes 6 weeks or 12 weeks to get a model to transition from development to production . . . Or moving from a laptop to the cloud, and to go from single machine to scaling — these are expensive handoffs often involve rewriting a bunch of code."

Learn more:

Anyscale

Sky Computing Lab

Ray

Follow everyone on X:

Robert Nishihara

Jennifer Li

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this archive episode from 2015, a16z's Sonal Chokshi, Frank Chen, and Steven Sinofsky discuss DeepMind's breakthrough AlphaGo system, which mastered the ancient Chinese game Go and introduced the public to reinforcement learning.

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of the AI + a16z podcast, Luma Chief Scientist Jiaming Song joins a16z General Partner Anjney MIdha to discuss Jiaming's esteemed career in video models, culminating thus far in Luma's recently released Dream Machine 3D model that shows abilities to reason about the world across a variety of aspects. Jiaming covers the history of image and video models,  shares his vision for the future of multimodal models, and explains why he thinks Dream Machine demonstrates its emergent reasoning capabilities. In short: Because it was trained on a volume of high-quality video data that, if measured in relation to language data, would amount to hundreds of trillions of tokens.

Here's a sample of the discussion, where Jiaming explains the "bitter lesson" as applied to training generative models, and in the process sums up a big component of why Dream Machine can do what it does by using context-rich video data:

"For a lot of the problems related to artificial intelligence, it is often more productive in the long run to use methods that are simpler but use more compute, [rather] than trying to develop priors, and then trying to leverage the priors so that you can use less compute.

"Cases in this question first happened in language, where people were initially working on language understanding, trying to use grammar or semantic parsing, these kinds of techniques. But eventually these tasks began to be replaced by large language models. And a similar case is happening in the vision domain, as well . . . and now people have been using deep learning features for almost all the tasks. This is a clear demonstration of how using more compute and having less priors is good.

"But how does it work with language? Language by itself is also a human construct. Of course, it is a very good and highly compressed kind of knowledge, but it's definitely a lot less data than what humans take in day to day from the real world . . . 

"[And] it is a vastly smaller data set size than visual signals. And we are already almost exhausting the . . . high-quality language sources that we have in the world. The speed at which humans can produce language is definitely not enough to keep up with the demands of the scaling laws. So even if we have a world where we can scale up the compute infrastructure for that, we don't really have the infrastructure to scale up the data efforts . . .

"Even though people would argue that the emergence of large language models is already evidence of the scaling law . . . against the rule-based methods in language understanding, we are arguing that language by itself is also a prior in the face of more of the richer data signal that is happening in the physical world."

Learn more:

Dream Machine

Jiaming's personal site

Luma careers

The bitter lesson

Follow everyone on X:

Jiaming Song

Anjney Midha

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode, design engineer Alasdair Monk joins a16z's Yoko Li and Derrick Harris to discuss how generative AI is changing how developers — and the those building for developers — interact with the tools of their trade. Alasdair’s journey includes stints at dev-centric companies such as Heroku/Salesforce, and he's presently designing the user experience for Poolside, an AI programming startup. 

Here's a sample of Alasdair discussing the future of the prompt bar in generative coding tools:

 "When interacting with machine learning models, we've almost thrown away 30 years of human-computer interaction knowledge and kind of reverted to using a terminal circa 1980 to interact with computer, or the prompt bar. This very plain-text way to interact with AI is really interesting. 

"I think it's very different when you can't predict what a user interface is going to look like. What an LLM can spit out is basically unpredictable or non-deterministic, and so how do you design for that or how do you design around the guardrails for that are the really interesting things that I think everyone who works in the industry right now is trying to figure out. And I think it's pretty clear to a lot of people that sometimes you want to chat to the computer as if it's like the rubber duck.

"I think a lot of where AI is going to really help us, particularly with engineering, is going to be in the interactions that aren't that at all, and will actually probably look much more like interacting with traditional software today, where I interact with it via windows and buttons and all sorts of GUI elements."

Follow everyone on X:

Alasdair Monk

Yoko Li

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode, Inngest cofounder and CEO Tony Holdstock-Brown joins a16z partner Yoko Li, as well as Derrick Harris, to discuss the reality and complexity of running AI agents and other multistep AI workflows in production. Tony also why developer tools for generative AI — and their founders — might look very similar to previous generations of these products, and where there are opportunities for improvement.

Here's a sample of the discussion, where Tony shares some advice for engineers looking to build for AI:

"We almost have two parallel tracks right now as, as engineers. We've got the CPU track in which we're all like, 'Oh yeah, CPU-bound, big O notation. What are we doing on the application-level side?' And then we've got the GPU side, in which people are doing like crazy things in order to make numbers faster, in order to make differentiation better and smoother, in order to do  gradient descent in a nicer and more powerful way. The two disciplines right now are working together, but are also very, very, very different from an engineering point of view. 

"This is one interesting part to think about for like new engineers, people that are just thinking about what to do if they want to go into the engineering field overall. Do you want to be on the side using AI, in which you take all of these models, do all of this stuff, build the application-level stuff, and chain things together to build products? Or do you want to be on the math side of things, in which you do really low-level things in order to make compilers work better, so that your AI things can run faster and more efficiently? Both are engineering, just completely different applications of it."

Learn more:

The Modern Transactional Stack

The LLM App Stack

Follow everyone on X:

Tony Holdstock-Brown

Yoko Li

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode, Ideogram CEO Mohammad Norouzi joins a16z General Partner Jennifer Li, as well as Derrick Harris, to share his story of growing up in Iran, helping build influential text-to-image models at Google, and ultimately cofounding and running Ideogram. He also breaks down the differences between transformer models and diffusion models, as well as the transition from researcher to startup CEO.

Here's an excerpt where Mohammad discusses the reaction to the original transformer architecture paper, "Attention Is All You Need," within Google's AI team:

"I think [lead author Asish Vaswani] knew right after the paper was submitted that this is a very important piece of the technology. And he was telling me in the hallway how it works and how much improvement it gives to translation. Translation was a testbed for the transformer paper at the time, and it helped in two ways. One is the speed of training and the other is the quality of translation. 

"To be fair, I don't think anybody had a very crystal clear idea of how big this would become. And I guess the interesting thing is, now, it's the founding architecture for computer vision, too, not only for language. And then we also went far beyond language translation as a task, and we are talking about general-purpose assistants and the idea of building general-purpose intelligent machines. And it's really humbling to see how big of a role the transformer is playing into this."

Learn more:
Investing in Ideogram

Imagen

Denoising Diffusion Probabilistic Models

Follow everyone on X:

Mohammad Norouzi

Jennifer Li

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

For this holiday weekend (in the United States) episode, we've stitched together two archived episodes from the a16z Podcast, both featuring General Partner Anjney Midha. In the first half, from December, he speaks with Mistral cofounder and CEO Arthur Mensch about the importance of open foundation models, as well as Mistral's approach to building them. In the second half (at 34:40), from February, he speaks with Stanford's Stefano Ermon about the state of the art in video models, including how OpenAI's Sora might work under the hood.

Here's a sample of what Arthur had to say about the debate over how to regulate AI models:

"I think the battle is for the neutrality of the technology. Like a technology, by a sense, is something neutral. You can use it for bad purposes. You can use it for good purposes. If you look at what an LLM does, it's not really different from a programming language. . . .

"So we should regulate the function, the mathematics behind it. But, really, you never use a large language model itself. You  always use it in an application, in a way, with a user interface. And so,  that's the one thing you want to regulate. And what it means is that companies like us, like foundational model companies, will obviously make the model as controllable as possible so that the applications on top of it can be compliant, can be safe. We'll also build the tools that allow you to measure the compliance and the safety of the application, because that's super useful for the application makers. It's actually needed.  

"But there's no point in regulating something that is neutral in itself, that is just a mathematical tool. I think that's the one thing that we've been hammering a lot, which is good, but there's still a lot of effort in making this strong distinction, which is super important to understand what's going on."

Follow everyone on X:

Anjney Midha

Arthur Mensch

Stefano Ermon

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

a16z partners Guido Appenzeller and Matt Bornstein join Derrick Harris to discuss the state of the generative AI market, about 18 months after it really kicked into high gear with the release of ChatGPT — everything from the emergence of powerful open source LLMs to the excitement around AI-generated music.

If there's one major lesson to learn, it's that although we've made some very impressive technological strides and companies are generating meaningful revenue, this is still a a very fluid space. As Matt puts it during the discussion:

"For nearly all AI applications and most model providers,  growth is kind of a sawtooth pattern, meaning when there's a big new amazing thing announced, you see very fast growth.  And when it's been a while since the last release, growth kind of can flatten off. And you can imagine retention can be  all over the place, too . . .

"I think every time we're in a flat period, people start to think, 'Oh, it's mature now,  the, the gold rush is over. What happens next?' But then a new spike almost always comes, or at least has over the last 18 months or so. So a lot of this depends on your time horizon, and I think we're still in this period of, like, if you think growth has slowed, wait a month  and see it change."

Follow everyone on X:

Guido Appenzeller

Matt Bornstein

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this bonus episode, recorded live at our San Francisco office, security-startup founders Dean De Beer (Command Zero), Kevin Tian (Doppel), and Travis McPeak (Resourcely) share their thoughts on generative AI, as well as their experiences building with LLMs and dealing with LLM-based threats.

Here's a sample of what Dean had to say about the myriad considerations when choosing, and operating, a large language model:

"The more advanced your use case is, the more requirements you have, the more data you attach to it, the more complex your prompts — ll this is going to change your inference time. 

"I liken this to perceived waiting time for an elevator. There's data scientists at places like Otis that actually work on that problem. You know, no one wants to wait 45 seconds for an elevator, but taking the stairs will take them half an hour if they're going to the top floor of . . . something. Same thing here: If I can generate an outcome in 90 seconds, it's still too long from the user's perspective, even if them building out and figuring out the data and building that report [would have] took them four hours . . . two days."

Follow everyone:

Dean De Beer

Kevin Tian

Travis McPeak

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode of the AI + a16z podcast, a16z General Partner Zane Lackey and a16z Partner Joel de la Garza sit down with Derrick Harris to discuss how generative AI — LLMs, in particular — and foundation models could effect profound change in cybersecurity. After years of AI-washing by security vendors, they explain why the hype is legitimate this time as AI provides  a real opportunity to help security teams cut through the noise and automate away the types of drudgery that lead to mistakes.

"Often when you're running a security team, you're not only drowning in noise, but you're drowning in just the volume of things going on," Zane explains. "And so I think a lot of security teams are excited about, 'Can we utilize AI and LLMs to really take at least some of that off of our plate?'

"I think it's still very much an open question of how far they go in helping us, but even taking some meaningful percentage off of our plate in terms of overall work is going to really help security teams overall."

Follow everyone:

Zane Lackey

Joel de la Garza

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

Socket Founder and CEO Feross Aboukhadijeh joins a16z's Joel de la Garza and Derrick Harris to discuss the open-source software supply chain. Feross and Joel share their thoughts and insights on topics ranging from the recent XZutils attack to how large language models can help overcome understaffed security teams and overwhelmed developers. 

Despite some increasingly sophisticated attacks making headlines and compromising countless systems, they're optimistic that LLMs, in particular, could be a turning point for security blue teams. As Feross sums up one possibility:

"The way we think about gen AI on the defensive side is that it's not as good as a human looking at the code, but it's something. . . . Our challenge is that we want to scan all the open source code that exists out there. That is not something you can pay humans to do. That is not scalable at all. But, with the right techniques, with the right pre-filtering stages, you can actually put a lot of that stuff through LLMs and out the other side will pop a list of of risky packages.

"And then that's a much smaller number that you can have humans take a look at. And so we're using it as a tool . . . to find the needle in the haystack, what is worth looking at. It's not perfect, but it can help cut down on the noise and it can even make this problem tractable, which previously wasn't even tractable."

More about Socket and  cybersecurity:

Socket

Investing in Socket

Hiring a CISO

Follow everyone :

Feross Aboukhadijeh

Joel de la Garza

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

In this episode, though, we’re traveling back in time to distant — in AI years, at least — past of 2020. Because amid all the news over the past 18 or so months, it’s easy to forget that generative AI — and LLMs, in particular — have been around for a while. OpenAI released its GPT-2 paper in late 2018, which excited the AI research community, and in 2020 made GPT-3 (as well as other capabilities) publicly available for the first time via its API. This episode dates back to that point in time (it was published in July 2020), when GPT-3 piqued the interest of the broader developer community and people really started testing what was possible.

And although it doesn’t predict the precambrian explosion of multimodal models, regulatory and copyright debate, and entrepreneurial activity that would hit a couple of years later — and who could have? — it does set the table for some of the bigger — and still unanswered — questions about what tools like LLMs actually mean from a business perspective. And, perhaps more importantly, what they ultimately mean for how we define intelligence.

So set your wayback machine to the seemingly long ago summer of 2020 and enjoy a16z’s Sonal Chokshi and Frank Chen discussing the advent of commercially available LLMs.

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

Pinecone Founder and CEO Edo Liberty joins a16z's Satish Talluri and Derrick Harris to discuss the promises, challenges, and opportunities for vector databases and retrieval augmented generation (RAG). He also shares insights and highlights from a decades-long career in machine learning, which includes stints running research teams at both Yahoo and Amazon Web Services.

Because he's been at this a long time,  and despite its utility, Edo understands that RAG — like most of today's popular AI concepts — is still very much a progress:

"I think RAG  today is where transformers were in 2017. It's clunky and weird and hard to get right. And it  has a lot of sharp edges, but it already does something amazing. Sometimes, most of the time, the very early adopters and the very advanced users are already picking it up and running with it and lovingly deal with all the sharp edges ...

"Making progress on RAG, making progress on information retrieval, and making progress on making AI more knowledgeable and less hallucinatory and more dependable, is a complete greenfield today. There's an infinite amount of innovation that will have to go into it."

More about Pinecone and RAG:

Investing in Pinecone

Retrieval Augmented Generation (RAG)

Emerging Architectures for LLM Applications

Follow everyone on X:

Edo Liberty

Satish Talluri

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

a16z General Partner Anjney Midha joins the podcast to discuss what's happening with hardware for artificial intelligence. Nvidia might have cornered the market on training workloads for now, but he believes there's a big opportunity at the inference layer — especially for wearable or similar devices that can become a natural part of our everyday interactions. 

Here's one small passage that speaks to his larger thesis on where we're heading:

"I think why we're seeing so many developers flock to Ollama is because there is a lot of demand from consumers to interact with language models in private ways. And that means that they're going to have to figure out how to get the models to run locally without ever leaving without ever the user's context, and data leaving the user's device. And that's going to result, I think, in a renaissance of new kinds of chips that are capable of handling massive workloads of inference on device.

"We are yet to see those unlocked, but the good news is that open source models are phenomenal at unlocking efficiency.  The open source language model ecosystem is just so ravenous."

More from Anjney:

The Quest for AGI: Q*, Self-Play, and Synthetic Data

Making the Most of Open Source AI

Safety in Numbers: Keeping AI Open

Investing in Luma AI

Follow everyone on X:

Anjney Midha

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

Naveen Rao, vice president of generative AI at Databricks, joins a16z's Matt Bornstein and Derrick Harris to discuss enterprise usage of LLMs and generative AI. Naveen is particularly knowledgeable about the space, having spent years building AI chips first at Qualcomm and then as the founder of AI chip startup Nervana Systems back in 2014. Intel acquired Nervana in 2016.

After a stint at Intel, Rao re-emerged with MosaicML in 2021. This time, he focused on the software side of things, helping customers train their own LLMs, and also fine-tune foundation models, on top of an optimized tech stack. Databricks acquired Mosaic in July of 2023.

This discussion covers the gamut of generative AI topics — from basic theory to specialized chips — to  although we focus on how the enterprise LLM market is shaping up. Naveen also shares his thoughts on why he prefers finally being part of the technology in-crowd, even if it means he can’t escape talking about AI outside of work.

More information:

LLMs at Databricks

Mosaic Research

More AI content from a16z

Follow everyone on X:

Naveen Rao

Matt Bornstein

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

There are few terms in the world of AI — if any — that invoke more of a reaction than a simple four-letter word: Open. Whether it’s industry debates over business models and the actual definition of open, or the US government actively discussing how to regulate open models, seemingly everyone has an opinion on what it means for AI models to be open. The good, the bad, and the ugly.

But to be fair, there’s good reason for this. In a world where many developers have come to expect open source tools at every level of the stack, the idea of powerful models locked behind enterprise licenses and corporate ethics can be disconcerting — especially for a technology as game-changing as AI promises to be. It’s a matter of who has the ability to innovate in the space, and whose release schedules and guardrails they’re beholden to.

This is why, back in February, a16z convened a panel of experts to discuss the state — and future — of open source AI models.

Featuring:

• Jim Zemlin (Executive Director, Linux Foundation)
• Mitchell Baker (Executive Chair, Mozilla Corp.)
• Percy Liang (Associate Professor, Stanford; Cofounder, Together AI)
• Anjney Midha (General Partner, a16z)
• Derrick Harris (Editorial Partner, a16z)

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

The AI + a16z podcast captures our thinking on AI across a broad swath of areas, from the infrastructure that powers today’s foundation models to how specific tools, like LLMs, are reshaping the hiring process. Looking forward, you can expect to hear about a list of topics that includes the latest advances in generative AI, cybersecurity, and the emerging stack of tools for building and running LLMs.Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.