Meta’s open source approach to AI puzzles Wall Street, techies love it

At Meta’s annual Connect conference last month, virtual reality enthusiasts gathered to hear about Mark Zuckerberg’s multibillion-dollar bet on the metaverse, the technology that’s supposed to define the company’s future.

But at this year’s event, VR developers were inundated with panel discussions about a topic that’s quickly becoming less about tomorrow and more about the present: artificial intelligence.

“Don’t tell Mark, but it feels less mixed reality and more AI these days,” joked Joseph Spisak, who joined the company as director of product development for generative AI two months earlier, during his session at Connect. “It kind of feels like an AI conference, which is kind of in my wheelhouse.”

Sandwiched between panels about Meta’s latest Quest 3 VR headset and augmented reality developer software were several sessions dedicated to Llama, Meta’s large language model (LLM) that’s gained popularity since OpenAI’s ChatGPT chatbot exploded onto the scene in November, sparking a sprint by leading tech companies to bring competitive offerings to market.

Zuckerberg, who changed Facebook’s name to Meta in late 2021 to signal his commitment to the metaverse, reminded Connect attendees that Llama was the power supply to the company’s latest digital assistants unveiled at the conference.

While Zuckerberg still views the growth of the nascent metaverse as critical to his company’s success, AI has emerged as the market he’s trying to win today. Meta views Llama and its family of generative AI software as the open source alternative to GPT, the LLM from Microsoft-backed OpenAI, and Google’s PaLM 2, which powers the search company’s Bard AI technology.

Industry experts compare Llama’s positioning in generative AI to that of Linux, the open source rival to Microsoft Windows, in the PC operating system market. Just as Linux software made its way into corporate servers worldwide and became a key piece of the modern internet, Meta sees Llama as the potential digital scaffolding supporting the next generation of AI apps.

On Wall Street, Llama is hard to value and, for many investors, hard to understand. Because AI researchers are at a premium and the infrastructure required to build and run models requires massive costs, Meta is investing heavily to build Llama, the updated Llama 2 that was introduced in July, and related generative AI software.

After the July announcement, Yann LeCun, the AI researcher Zuckerberg hired in 2013 to lead Facebook’s new AI research group, wrote on Twitter that, “This is going to change the landscape of the LLM market.”

But open source means Meta is giving away the software for free to developers, a dramatically different approach to the traditional software license and subscription models and far afield from the highly lucrative digital ad business that turned Facebook into an internet powerhouse.

In announcing Llama 2, Meta said the new version would have a commercial license that allows companies to integrate it into their products. The company has said it isn’t focused on monetizing Llama 2 directly, but it does earn an undisclosed amount of money from cloud-computing companies like Microsoft and Amazon, which offer access to Llama 2 as part of their own generative AI enterprise services.

Zuckerberg said on the company’s second-quarter earnings call that he doesn’t expect Llama 2 to generate “a large amount of revenue in the near term, but over the long term, hopefully that can be something.”

Meta is looking to benefit from Llama in other ways.

Zuckerberg told analysts in July that improvements made to Llama by third-party developers could result in “efficiency gains,” making it cheaper for Meta to run its AI software. Meta said it expects capital expenditures for 2023 to be in the range of $27 billion to $30 billion, down from $32 billion last year. Finance chief Susan Li said the figure will likely grow in 2024, driven in part by data center-and AI-related investments.

Influence brings its own advantages. If the world’s leading AI researchers use Llama, Meta could have an easier time hiring skilled technologists who understand the company’s approach to development. Facebook has a history of using open source projects, such as its PyTorch coding framework for machine learning apps, as a recruiting tool, luring technologists who want to work on cutting-edge software projects.

As Zuckerberg strives for efficiency, he’s got his eyes on Nvidia, which is generating billions of dollars in quarterly profits for its AI chips. Meta is one of its biggest customers. Jim Fan, a senior AI science at Nvidia, said in a post on X that it likely cost Meta $20 million to train Llama 2, considerably more than the estimated $2.4 million it took to train its predecessor.

Mainstream adoption of Llama 2 could influence Nvidia to ensure its graphics processing units (GPUs) work well with Meta-sanctioned software, lowering the company’s AI training and computing costs.

Meanwhile, Meta has its own internal AI chip projects, giving it a potential alternative to Nvidia’s processors.

“It gives them some price negotiating room,” said Arjun Bansal, CEO of enterprise startup Log10 and a former AI chip executive. “Nvidia wants to charge a lot and they can be like, ‘Hey, we got our own thing.'”

Nathan Lambert remembers the energy emanating from his colleagues at AI startup Hugging Face the weekend Meta debuted its much-anticipated Llama 2.

Lambert and his teammates worked overtime to ensure the company’s infrastructure was ready to handle the influx of coders looking to take Llama 2 for a test drive. 

Along with cloud-computing engines Microsoft Azure and Amazon Web Services, Hugging Face was one of Meta’s chosen launch partners for Llama 2, but arguably the most important. Developers, AI researchers and thousands of companies use Hugging Face’s platform to share code, data sets and models, making it one of the industry’s biggest communities.

Although a number of open source LLMs are available, Lambert said Llama 2 is by far the most popular.

“It’s the model that most people are playing with and that most startups are playing with,” said Lambert, who announced on Oct. 4 that he’s leaving Hugging Face though he didn’t say where he’s going.

As with all things Zuckerberg, the project is not without controversy. Some in the industry consider Meta’s licensing agreement to use Llama 2 as limiting, conflicting with the spirit of collaborative development and innovation.

For instance, third-party developers must request approval from Meta to use Llama 2 if they incorporate the software into any products or services that had “greater than 700 million monthly active users” in the month prior to its July release. Critics have said this clause was a way to keep rivals like Snap or TikTok from using Llama 2 for their own services.

“It’s pretty restrictive,” said Umesh Padval, a venture partner at Thomvest Ventures and investor in AI startup Cohere, which builds proprietary LLMs. “It looks like Meta wants all the benefits of open source for their business while keeping the competition away.”

Lambert said Meta could do itself a favor with the open source community and release more details about the specific, underlying datasets used to train Llama 2 so developers could better understand the training process. Open source adherents and privacy experts have pushed for more transparency into what kinds of data has been used to train LLMs, but companies have so far revealed few details.

“We believe in open innovation, and we do not want to place undue restrictions on how others can use our model,” a Meta spokesperson said in a statement. “However, we do want people to use it responsibly. This is a bespoke commercial license that balances open access to the models with responsibility and protections in place to help address potential misuse.”

Despite some detractors, Meta’s model is seeing plenty of early uptake. The company disclosed at Connect that there have been “more than 30 million downloads of Llama-based models through Hugging Face and over 10 million of these in the last 30 days alone.”

Nvidia’s Fan noted in his X post that Llama 2’s new commercial license could lure more companies to experiment with the language model compared to the original Llama.

“AI researchers from big companies were wary of Llama-1 due to licensing issues, but now I think many of them will jump on the ship and contribute their firepower,” Fan wrote.

As of today, businesses investing in AI prefer to use commercially available LLMs, according to a recent TC Cowen survey of 680 firms in cloud computing. The survey found that 32% of respondents have used or plan to use commercially packaged LLMs like OpenAI’s GPT-4 software while 28% were focused on open source LLMs like Llama and Falcon, developed in the United Arab Emirates. Only 12% of respondents planned on using in-house LLMs.

At the U.S. Government Accountability Office, Taka Ariga studies how bleeding-edge technologies like LLMs could help the agency better conduct audits and investigations through its Innovation Lab.

By the end of the year, Ariga’s team is planning to finish its first experiment investigating how LLMs can potentially be used to summarize numerous GAO reports and materials on a particular topic, and then combine those files with various other potentially relevant documentation from other agencies.

“The general public or a member of congress might say, ‘What has the GAO done in the area of nuclear safety?'” Ariga said, regarding the LLM project. “Of course, we have done a lot of work, but that’s sort of report-by-report basis; you can’t do that kind of sort of topical search.”

The GAO is currently using AWS’ Bedrock generative AI service to help the agency experiment with various popular LLMs, including proprietary models offered by startups like Cohere and Anthropic.

While AWS recently said Bedrock will soon support Llama 2, Ariga said the GAO is first testing Anthropic’s Claude LLM and will likely pass on using Llama 2 because of Meta’s poor reputation in Washington.