Meta isn’t going to just sit back and watch OpenAI run away with the AI trophy. Quite the opposite, actually, based on yesterday’s announcement. ICYMI: Mark Zuckerberg, or should I say “Zuck,” took to Instagram Live to announce Llama 3, the company’s largest suite of AI models to date.
So what’s the big deal? Well, a few things. First is the size of the models. In total, there are three of them. The two smallest, Llama 8B and 70B, contain 8 billion and 70 billion parameters (settings that guide AI's learning process), respectively. A third and much larger model, Llama 400B, contains 400 billion parameters. Yes, you read that right.
Second is the amount of data these models are trained on. According to the company announcement, Llama 3 models were trained on over 15 trillion tokens (specific units like words that models are trained on) while the previous generation, Llama 2, was trained on 2 trillion tokens — a respectable amount, for sure, but eclipsed by today’s model.
Third is how it compares to other models on the market. Meta claims Llama 8B and 70B are “among the best-performing models for their respective parameter count.” That’s a big claim to make, so how does it actually stack up?
Meta points to Llama 3’s score across different AI benchmarks like MMLU and ARC, which test models based on a number of questions ranging from biology to mathematics. According to those scores, Llama 8B beats the likes of Mistral 7B and Gemini 7B, while Llama 70B beats Gemini 1.5 Pro and comes up just short of Anthropics most powerful model, Claude 3 Opus.
Like previous models, Llama 3 is entirely open-source, so developers can contribute to its direction and improvements. Who knows, maybe we will crowdsource AGI?
If you want to test out Llama 3, head over to Meta’s standalone AI website where you can chat directly with the currently available models (8B and 70B) as long as you’re in a country where it’s currently available. Alongside that, you can try out Meta’s AI assistant directly on Instagram now.
123 •
Web App
