“/>Photo credit: v_alex / Getty Images
Developing cutting-edge AI systems like ChatGPT requires massive technical resources, in part because they are costly to develop and operate. While several open-source efforts have attempted to reverse-engineer proprietary closed-source systems created by commercial labs like Alphabet’s DeepMind and OpenAI, they have often encountered obstacles — largely due to a lack of capital and expertise.
Hoping to avoid that fate, a community research group, EleutherAI, is establishing a charitable foundation. The organization announced today that it will launch a nonprofit research institute, the EleutherAI Institute, funded by donations and grants from supporters including AI startups Hugging Face and Stability AI, former GitHub CEO Nat Friedman, Lambda Labs and Canvas.
“Formalizing it as an organization allows us to build full-time staff and engage in longer and more involved projects than would be possible as a volunteer group,” Stella Biderman, AI researcher at Booz Allen Hamilton, who will co-lead the EleutherAI Institute, told TechCrunch in an email interview. “Specifically in terms of a non-profit organization, I think it’s a no-brainer given that we’re focused on research and the open source space.”
EleutherAI started a few years ago as a grassroots collection of developers working on open source AI research. Its founding members — Connor Leahy, Leo Gao, and Sid Black — wrote the code and collected the data needed to create a machine learning model that approximates OpenAI’s text-generating GPT-3, which was getting a lot of attention at the time aroused.
The company curated and published The Pile, a collection of datasets used to train GPT-3-like models to complete text, write code, and more. And it released several models under the Apache 2.0 license, including GPT-J and GPT-NeoX, language models that fueled a whole new wave of startups for a while.
To train its models, EleutherAI relied primarily on the TPU Research Cloud, a Google Cloud program that supports projects with the expectation that the results will be shared publicly. CoreWeave, a US-based cryptocurrency miner that provides cloud services for AI workloads, also provided computing resources to EleutherAI in exchange for models that its customers can use and operate.
EleutherAI grew rapidly. Today, over 20 of the community’s regular contributors work full-time and primarily focus on research. And in the past 18 months, EleutherAI members have co-authored 28 papers, trained dozens of models, and released ten codebases.
But the fickle nature of its cloud providers sometimes forced EleutherAI to scrap its plans. Initially, the group had intended to release a model roughly the size of GPT-3 in terms of the number of parameters, but has shelved that roadmap for technical and financial reasons. (In AI, parameters are the parts of the model learned from historical training data and essentially define the model’s capabilities when faced with a problem, such as generating text.)
At the end of 2022, EleutherAI became acquainted with Stability AI, the now well-financed startup behind the image-generating AI system Stable Diffusion. Along with other collaborators, it helped create the first version of Stable Diffusion. And since then, Stability AI has donated some of the computing power from its AWS cluster to EleutherAI’s ongoing language model research.
After another big funder — Hugging Face — reached out to EleutherAI, discussions about nonprofits began, says Biderman. (Many EleutherAI employees were involved in the company’s BigScience effort, which aimed to train and open source a GPT-3-like model over the course of a year.)
“EleutherAI has historically and likely will continue to focus largely on large language models that are architecturally similar to ChatGPT,” Biderman said. “Beyond training large language models, we are excited to provide more resources for ethics, interpretability, and alignment work.”
One might wonder whether the involvement of commercially motivated companies like Stability AI and Hugging Face – both of which are backed by significant venture capital – could influence EleutherAI’s research. It’s a natural assumption – and it’s even supported by evidence. At least one study shows a direct link between donations and the likelihood that nonprofits will comment on a proposed government rule.
Biderman asserts that the EleutherAI Foundation will remain independent and sees no problem with the donor pool so far.
“We don’t develop models on behalf of commercial companies,” Biderman said. “If anything, I think diverse sponsorship enhances our independence. If we were fully funded by a tech company, that seems like a much bigger potential problem on our part.”
Another challenge the EleutherAI Foundation faces is making sure their coffers don’t run dry. OpenAI is a cautionary tale; Founded as a not-for-profit organization in 2015, the company later transitioned to a capped-profit structure to fund its ongoing research.
By and large, nonprofit initiatives funding AI research have been a mixed bag.
Success stories include the Allen Institute for AI (AI2), founded by the late Microsoft co-founder Paul Allen, which aims to achieve scientific breakthroughs in the fields of AI and machine learning. There’s also the Alan Turing Institute, the UK-based, government-funded national institute for data science and machine learning. Smaller promising efforts include Cohere For AI by AI startup Cohere (despite its corporate ties) and Distributed AI Research by Timnit Gebru, a globally distributed research organization.
But for every AI2, there’s former Google Chairman Eric Schmidt’s AI Research Fund. Worth over $125 million, it sparked new controversy after Politico reported that Schmidt wields unusually powerful influence over the White House Office of Science and Technology Policy.
Time will tell which direction the EleutherAI Foundation ultimately takes. It is likely that the mission will evolve and change over time – in a positive way, we can only hope.
