Llama - 3.1-405B: an open source model to rival GPT-4o / Claude-3.5

This could be The One - the open source model that closes the gap with the top closed models like GPT-4o / Claude-3.5. It's a "curves-crossing" moment reminiscent of how the Intel vs. ARM approaches played out, and perhaps with similar profound effects on the landscape. If you're in SF, join us tonight for a meetup and 405B panel including founders of Vercel and JuliusAI: https://lu.ma/4es9bfgs Also, one-time only: Launch your product TODAY using Llama-3 405b and we'll feature it (this won't prevent you from launching in the near future). Plus the top launches will be eligible to demo tonight after the panel. In private conversations with launch partners, Meta has emphasized 405B's reasoning capability and multilingual abilities. This would seem to have big implications for interfaces, especially voice interfaces. Are people finding the model lives up to this, practically? Som more thoughts from a friend, @kwindla (Daily.co), who is a launch partner for 405B: "1. 405B beats GPT-4o on 11 of 13 widely used benchmarks. And Meta/Fair has a history of being careful about these benchmarks, so they almost certainly went to a lot of effort to not let training data leak into test, etc. No open source model has previously come close to GPT-4o/Claude-3.5. It’s a huge, huge deal if this is accurate and reflects the quality of “reasoning” the model can do. 2. The two smaller 3.1 models (70B and 8B) also made big leaps in benchmark performance. That indicates that Meta’s strategy for training/distilling is working. Having models that are small enough to run on single devices (or, on LPUs, very very very fast and inexpensively) that are this good may be equivalent to leap-frogging GPT-4o-mini. This also gives people the opportunity to experiment with fine-tuning really good models and with doing architecture/merge experiments. 3. Big models have a different “tone/vibe” than small models. 3-70B was a pretty good model in a lot of ways, but as a conversational agent it just didn’t feel as good qualitatively as GPT-4o and Claude-3.5. That feel really matters in things like consumer-facing voice chat use cases. If 405B is approximately as good as the proprietary models on benchmarks, *and* matches their “vibe” for the first time, that’s truly exciting for a whole range of next-generation conversational/interactive use cases."

This is something serious that can be a game-changer. I am surprised at the votes LOL - maybe the voting mechanism truly need some changes.

A different perspective for product launch. But is it limited to a selected ones? or its available for all ?

Wait a minute. Llama 3.1 can't be called "open-source." It looks impressive, and I’m excited to test it. But it doesn't fit the proposed definitions of "open-source AI." For LLMs to be truly "open-source," they need to include: 1: Detailed info about the training data so that someone can recreate a similar system with similar data. 2: The source code used to train and run the system. This is an important requirement, not a simple detail. While "open-source" has clear definitions in traditional software, its definition is still a work in progress for AI. Concepts are evolving, but we need to be more careful with terms and separate PR buzz from reality. For more on the definition of Open Source AI and this debate, check out: https://opensource.org/deepdive/...

Of course this happens on the day I launch!! jk jk - this is super cool - good job meta

Wow, interesting. Does anyone know how much it costs to be featured here?

Wow, exciting news and we were using Llama 3 and now we will move to Llama 3.1

Good hunt Rajiv! The best of the best AI models will be the ones that are open source.

Great competition is great news for the market, especially if it's open source. I'm super stoked to try it out, especially self-hosting the smaller models. Congrats to the team at Meta :)

This is a riot! So excited!