And he wants Meta to open source it. Eventually. Maybe.
Fueling the generative AI craze is a view that the tech industry is on a path to getting superhuman, god-like intelligence.
OpenAI’s stated goal is to build this Artificial General Intelligence , or AGI. Demis Hassabis, the leader of Google’s AI projects, has the same goal.
Now, Meta CEO Mark Zuckerberg is joining the game. While he doesn’t have a timeline for when artificial general intelligence (AGI) will be reached, or even a clear description for it, he wants to build it. At the same time, he’s shaking things up by moving Meta’s AI study group, FAIR, to the same part of the company as the team building creative AI products across Meta’s apps. The goal is for Meta’s AI breakthroughs to more directly reach its billions of users.
“We’ve come to this view that, in order to build the products that we want to build, we need to build for general intelligence,” Zuckerberg tells me in an exclusive conversation. “I think that’s important to convey because a lot of the best researchers want to work on the more ambitious problems.”
Here, Zuckerberg is saying the quiet part loudly. The fight for AI talent has never been more fierce, with every company in the space vying for an extremely small group of researchers and engineers. Those with the needed knowledge can earn eye-popping compensation packages to the tune of over $1 million a year. CEOs like Zuckerberg are regularly pulled in to try to win over a key hire or keep a researcher from defecting to a rival.
“We’re used to there being pretty intense talent wars,” he says. “But there are different dynamics here with multiple companies going for the same profile, [and] a lot of VCs and folks throwing money at different projects, making it easy for people to start different things externally.”
After ability, the scarcest resource in the AI field is the computing power needed to train and run big models. On this topic, Zuckerberg is ready to flex. He tells me that, by the end of this year, Meta will own more than 340,000 of Nvidia’s H100 GPUs — the industry’s chip of choice for building creative AI.
Mark Zuckerberg’s Artificial General Intelligence
“We have built up the capacity to do this at a scale that may be larger than any other individual company”
External research has put Meta’s H100 sales for 2023 at 150,000, a number that is tied only with Microsoft’s exports and at least three times bigger than everyone else’s. When its Nvidia A100s and other AI chips are accounted for, Meta will have a stockpile of almost 600,000 GPUs by the end of 2024, according to Zuckerberg.
“We have built up the capacity to do this at a scale that may be larger than any other individual company,” he says. “I think a lot of people may not appreciate that.”
The realization
No one working on AI, including Zuckerberg, seems to have a clear meaning for AGI or an idea of when it will arrive.
“I don’t have a one-sentence, pithy definition,” he tells me. “You can quibble about if general intelligence is akin to human level intelligence, or is it like human-plus, or is it some far-future super intelligence. But to me, the important part is actually the breadth of it, which is that intelligence has all these different skills where you have to be able to think and have intuition.”
He sees its final arrival as being a gradual process, rather than a single time. “I’m not actually that sure that some specific threshold will feel that profound.”
As Zuckerberg puts it, Meta’s new, broader focus on AGI was inspired by the release of Llama 2, its latest large language model, last year. The company didn’t think that the ability for it to create code made sense for how people would use an LLM in Meta’s apps. But it’s still an important skill to learn for making smarter AI, so Meta built it anyway.
“One hypothesis was that coding isn’t that important because it’s not like a lot of people are going to ask coding questions in WhatsApp,” he says. “It turns out that coding is actually really important structurally for having the LLMs be able to understand the rigor and hierarchical structure of knowledge, and just generally have more of an intuitive sense of logic.”
“Our ambition is to build things that are at the state of the art and eventually the leading models in the industry”
Meta is training Llama 3 now, and it will have code-generating skills, he says. Like Google’s new Gemini model, another focus is on more advanced thinking and planning skills.
“Llama 2 wasn’t an industry-leading model, but it was the best open-source model,” he says. “With Llama 3 and beyond, our ambition is to build things that are at the state of the art and eventually the leading models in the industry.”
Open versus closed
The question of who gets to finally control AGI is a highly discussed one, as the near implosion of OpenAI recently showed the world.
Zuckerberg holds total power at Meta thanks to his vote control over the company’s stock. That puts him in a particularly strong situation that could be dangerously enhanced if AGI is ever achieved. His answer is the plan that Meta has followed so far for Llama, which can — at least for most use cases — be called open source.
“I tend to think that one of the bigger challenges here will be that if you build something that’s really valuable, then it ends up getting very concentrated,” Zuckerberg says. “Whereas, if you make it more open, then that addresses a large class of issues that might come about from unequal access to opportunity and value. So that’s a big part of the whole open-source vision.”
Without naming names, he compares Meta’s method to that of OpenAI’s, which began with the aim of open sourcing its models but has becoming increasingly less visible. “There were all these companies that used to be open, used to publish all their work, and used to talk about how they were going to open source all their work. I think you see the dynamic of people just understanding, ‘Hey, this is going to be a really important thing, let’s not share it.’”
While Sam Altman and others laud the safety benefits of a more closed approach to AI research, Zuckerberg sees a shrewd business play. Meanwhile, the models that have been launched so far have yet to cause catastrophic damage, he says.
“The biggest companies that started off with the biggest leads are also, in a lot of cases, the ones calling the most for saying you need to put in place all these guardrails on how everyone else builds AI,” he tells me. “I’m sure some of them are legitimately concerned about safety, but it’s a hell of a thing how much it lines up with the strategy.”
“I’m sure some of them are legitimately concerned about safety, but it’s a hell of a thing how much it lines up with the strategy”
Zuckerberg has his own motives, of course. The end result of his open vision for AI is still a gathering of power, just in a different shape. Meta already has more users than almost any company on Earth and a wildly profitable social media business. AI features can possibly make his tools even stickier and more useful. And if Meta can successfully standardize the growth of AI by releasing its models publicly, its power over the ecosystem will only grow.
There’s another wrinkle: If AGI is ever achieved at Meta, the call to open source it or not is ultimately Zuckerberg’s. He’s not ready to commit either way.
“For as long as it makes sense and is the safe and responsible thing to do, then I think we will generally want to lean towards open source,” he says. “Obviously, you don’t want to be locked into doing something because you said you would.”
Don’t call it a turn
In the wider context of Meta, the timing of Zuckerberg’s new AGI push is a bit strange.
It has been only two years since he changed the company name to focus on the metaverse. Meta’s new smart glasses with Ray-Ban are showing early success, but full-fledged AR glasses feel increasingly further out. Apple, meanwhile, has recently confirmed his bet on headsets with the launch of the Vision Pro, even though VR is still a niche business.
Zuckerberg, of course, disagrees with the notion of his focus on AI being a shift.
“I don’t know how to more unequivocally state that we’re continuing to focus on Reality Labs and the metaverse,” he tells me, pointing to the fact that Meta is still spending north of $15 billion a year on the project. Its Ray-Ban smart glasses recently added a visual AI assistant that can spot items and speak languages. He sees generative AI having a more important part in Meta’s hardware work going forward.
“I don’t know how to more unequivocally state that we’re continuing to focus on Reality Labs and the metaverse”
He sees a future in which virtual worlds are created by AI and filled with AI characters that follow real people. He says a new platform is coming this year to let anyone build their own AI characters and spread them across Meta’s social apps. Perhaps, he says, these AIs will even be able to post their own content to the feeds of Facebook, Instagram, and Threads.
Meta is still a metaverse company. It’s the biggest social media company in the world. It’s now trying to build AGI. Zuckerberg frames all this around the overall goal of “building the future of connection.”
To date, that link has been mostly people interacting with each other. Talking to Zuckerberg, it’s clear that, moving forward, it’s increasingly going to be about people talking to AIs, too. It’s clear that he views this future as inevitable and exciting, whether the rest of us are ready for it or not.
