Meta Is Already Training a More Powerful Successor to Llama 3

On Thursday morning, Meta released its latest artificial intelligence model, Llama 3, touting it as the most powerful to be made open source so that anyone can use it. The same afternoon, Yann LeCun, Meta’s chief AI scientist, said an even more powerful successor to Llama is in the works. He suggested it could potentially outshine the world’s best closed AI models, including OpenAI’s GPT-4 and Google’s Gemini.

Meta released two versions of Llama 3 today, one with 8 billion parameters—an industry term that roughly conveys a model’s power—and another with 70 billion parameters. LeCun said that bigger models are in the works and that the most powerful, with more than 400 billion parameters, is currently in training.

“It takes a lot of time to fine-tune, but a bunch of variations of these models are going to come out in the next few months,” LeCun said at Imagination in Action, a conference at MIT focused on generative AI. It’s unclear when the model with more than 400 billion parameters might be released.

Meta on Tuesday launched a new assistant called Meta AI based on Llama 3 and says its testing shows that the models are better than previous open source models with similar numbers of parameters. Although most of the biggest AI developers like Google and OpenAI keep their technology closed, LeCun predicted that open source AI models will advance more rapidly. In theory, he said, they’ll push AI toward human-level intelligence more quickly.

LeCun argued that looking at the wider software industry shows that open source technology ultimately wins out. The open approach has become predominant in software infrastructure, he said, because it allows improvements to be shared more rapidly and code to be scrutinized more comprehensively. “AI is better when more people look at the code,” he said. “Infrastructure needs to be open source—it just progresses faster.”

Meta’s open source AI strategy has helped accelerate the current frenzy around generative AI. When the company released its Llama 2 model in July 2023 it provided many startups, researchers, and entrepreneurs access to much more powerful AI models to download to experiment with and build upon.

OpenAI’s GPT-4 and