Stability AI Launches the First of its StableLM Suite of Language Models
Stability AI Launches the First of its StableLM Suite of Language Models
Stability AI has unveiled its latest offering, StableLM, an open-source language model that is now available in an Alpha version with 3 billion and 7 billion parameters. The company plans to roll out a larger version that has between 15 billion and 65 billion parameters.
Developers are free to examine, use and customise the StableLM base models for both research and commercial purposes, subject to the terms of the CC BY-SA-4.0 licence.
Stability AI's previous release was the Stable Diffusion. It is a transparent, scalable and open image model that was made public in 2022. With StableLM, the company aims to make foundational AI technology accessible to everyone by providing efficient models that deliver remarkable performance with the right training. StableLM can produce text and code and is poised to support a variety of downstream applications.
Stability AI's release of StableLM is an extension of their previous work with EleutherA, a nonprofit research organisation. GPT-J, GPT-NeoX, and Pythia suite, all of which were trained on The Pile open-source dataset, were part of their language models that have been shared with the public. Cerebras-GPT and Dolly-2 are among the models that were developed from these initial efforts.
StableLM, on the other hand, was trained on a novel experimental dataset, which is three times larger than The Pile dataset, with 1.5 trillion tokens of content. Details about the dataset will be released in due course. Despite having only 3 to 7 billion parameters (compared to GPT-3's 175 billion parameters), StableLM's performance is outstanding in conversational and coding tasks due to the richness of the dataset.
In addition, we are making available a collection of research models that have undergone instruction fine-tuning. These models will use a blend of five current open-source datasets for conversational agents, namely Alpaca, GPT4All, Dolly, ShareGPT and HH. Please note that these fine-tuned models are strictly intended for research purposes and are being released under the non-commercial CC BY-NC-SA 4.0 licence, which aligns with Stanford's Alpaca licence.
Our aim is to ensure that language models serve as the foundation of our digital economy and are designed in a way that is inclusive of everyone. To this end, models like StableLM represent our dedication to AI technology that is transparent and supportive:
Transparency is critical to us, and we make our models open-source to foster trust and promote transparency. Researchers have the opportunity to scrutinise the models' performance, work on methods for interpretability, identify potential risks, and contribute to developing safeguards.
Furthermore, organisations across various sectors can tailor ("fine-tune") these open-source models to suit their specific requirements without compromising their confidential data or losing authority over their AI capabilities.
Our priority is to create language models that are accessible to the everyday user. Our focus on designing for the edge means that our models can run on local devices, enabling developers to create applications that work seamlessly with commonly available hardware. As a result, the economic benefits of AI can be shared by a broad community of users and developers, promoting greater collaboration and innovation.
Our models are also open-source, providing academics and researchers with fine-grained access to develop safety and interpretability techniques beyond what is possible with closed models. At the heart of our approach is the belief that AI technology should support, not replace, our users. We prioritise efficient, specialised, and practical AI performance, rather than the pursuit of god-like intelligence. Our tools are designed to empower individuals and businesses, unlocking creativity, boosting productivity, and creating new economic opportunities.
AI Catalog's chief editor
Share on social networks:
