“Llama Llama-3-405B?” by Zvi

It's here. The horse has left the barn. Llama-3.1-405B, and also Llama-3.1-70B and Llama-3.1-8B, have been released, and are now open weights.

Early indications are that these are very good models. They were likely the best open weight models of their respective sizes at time of release.

Zuckerberg claims that open weights models are now competitive with closed models. Yann LeCun says ‘performance is on par with the best closed models.’ This is closer to true than in the past, and as corporate hype I will essentially allow it, but it looks like this is not yet fully true.

Llama-3.1-405B not as good as GPT-4o or Claude Sonnet. Certainly Llama-3.1-70B is not as good as the similarly sized Claude Sonnet. If you are going to straight up use an API or chat interface, there seems to be little reason to use Llama.

That is a [...]

---

Outline:

(04:25) Options to Run It

(04:45) The Model Card

(08:42) Benchmarks

(13:41) Human Reactions in the Wild

(16:56) What's It Good For?

(21:39) The Other Other Guy

(22:35) Safety

(31:48) Three People Can Keep a Secret and Reasonably Often Do So

(36:12) The Announcement and Interview

(47:59) Zuckerberg's Open Weights Manifesto

(58:17) Fun Little Note

The original text contained 15 images which were described by AI.

---

First published:
July 24th, 2024

Source:
https://www.lesswrong.com/posts/fjzPg9ATbTJcnBZvg/llama-llama-3-405b

---

Narrated by TYPE III AUDIO.

---