Insights and research from the experts at MIT.

by Sara Brown, Senior News Editor

view online  /  subscribe

Businesses and employees are constantly making decisions, big and small, about how to use AI at work. But two recent research papers look at decisions that aren’t always on my radar: what resources companies put into building AI models, and how businesses decide what kind of models to use. These choices reflect broader themes about AI democratization and the future of the global AI economy. 

To defray the costs of training frontier models, some companies — such as OpenAI, Anthropic, and Google — keep them proprietary, or closed. That means users must pay to access the AI systems and to get responses from them. 

Open models, such as those from Meta, DeepSeek, and Mistral, make one or more model details public, like model weights, source code, training data, or architecture. That openness means that users can host and run models locally. 

In a new research paper, Frank Nagle, a researcher at the MIT Initiative on the Digital Economy, and Daniel Yue of Georgia Tech looked at the market for AI models. They found:  

Closed models are far more commonly used than open ones, accounting for nearly 80% of all AI tokens that are processed on OpenRouter, a widely used marketplace for AI models. That means open models account for only about 20% of AI tokens processed. (A token is a unit of input to or output from an AI model, roughly equivalent to one word in a prompt to an AI chatbot.)

But closed models cost significantly more to run — $1.86 per million tokens, on average, compared with 23 cents per million tokens for open models.

Open models are closing the gap in quality. Open models achieved about 90% of the performance of closed models when they were released, but they were usually able to close the gap within 13 weeks of a closed model’s initial release. 

Given all of the above, the researchers found that optimal reallocation of demand from closed to open models could cut average overall spending by more than 70%, saving the global AI economy about $25 billion annually. 

So why aren’t more users already using open models? The researchers pointed to valid concerns about the costs of switching to new models and reliability, regulatory, or security concerns that are easier to assuage with closed models. There are also misconceptions about inferior performance and a lack of data privacy when it comes to open models. 

Nagle encouraged companies to periodically review their use of AI models the same way they reevaluate software and infrastructure investments, and to consider whether there’s a more cost-effective way to meet their needs. 

“The difference between benchmarks is small enough that most organizations don’t need to be paying six times as much just to get that little bit of performance improvement,” Nagle said. “They need to think about how to use the right tool for the right job instead of defaulting to what’s popular.”

Another new paper looks at a different aspect of AI models and accessibility: the resources put into training and running AI models. Historically, companies have operated under the belief that bigger is better: as AI systems are given access to more data and computing power, they tend to improve. 

But that era might be over. Researchers Hans Gundlach, Jayson Lynch, and Neil Thompson from the MIT FutureTech lab found that what they call “meek models” — AI systems built or run with limited resources — could soon perform on par with today’s leading models while costing far less. (Consider the paper’s title: “Meek Models Shall Inherit the Earth.”)

Previous research has shown that as AI models grow, the performance gains from additional computing power start to fade, based on neural scaling laws. The MIT researchers found that companies will eventually see little comparative advantage from scaling their models much faster than other organizations, and models trained or run with limited resources (meek models) will perform more comparably to state-of-the-art models. 

This could upend how companies compete to build and use AI, as access to advanced systems becomes less about who can afford them and more about who can make the best strategic decisions. Companies that fine-tune models for specific uses or apply their own high-quality data will be better positioned than those building ever-bigger systems.

More people and organizations having access to high-performing models will also lower barriers to innovation and productivity gains. But the researchers also note that democratization gives more individuals access to dangerous capabilities and makes it harder to regulate AI use.

Researchers at the MIT Center for Information Systems Research outline how business models are evolving to keep pace with advances in AI. 

During last month’s World Economic Forum annual meeting in Davos, Switzerland, MIT Sloan principal research scientist Andrew McAfee moderated a discussion about how AI, compute, robotics, materials, and energy investments will determine which countries can sustain future growth.

At the recent MIT Sloan CFO Summit, experienced chief financial officers detailed how they’re using AI for forecasting, budgeting, and automating repetitive tasks, helping their teams focus on more meaningful, strategic work. 

MIT Sloan professor Catherine Tucker talked about how AI is changing advertising on the internet on an episode of Marketplace. 

Questions or comments?

Subscribe.

mitsloan.mit.edu

One Main Street 9th Floor | Cambridge, MA 02139 US
© 2024 MIT Sloan School of Management

Unsubscribe    Privacy Policy