Stability AI's Stable LM 2 With 1.6 Billion Parameter Is Here

The competition for the most powerful and accessible language models is heating up, especially in the realm of mobile-ready AI. In recent months, tech giants like Meta (Llama 2), Microsoft (Phi 2), and Google (Gemini Nano) have released their own small language models capable of running on smartphones and tablets.

Today, Stability AI joins the party with the release of Stable LM 2 with 1.6 billion parameters and multilingual capabilities.

What is Stable LM 2?

Stable LM 2 is a state-of-the-art 1.6 billion parameter small language model trained on multilingual data in English, Spanish, German, Italian, French, Portuguese, and Dutch.

Stable LM 2 stands shoulder-to-shoulder with other prominent models in its size category, outperforming Microsoft’s Phi-1.5 (1.3B) and Phi-2 (2.7B), TinyLlama 1.1B, and Falcon 1B.

Stability AI stands out for its commitment to transparency. They have released the pre-training checkpoint, allowing developers to fine-tune the model for specific needs.

Additionally, they are committed to providing full training data details to facilitate reproducibility. This level of openness is a welcome step forward in the field of AI development.

Things to know about Stable LM 2

• It is trained on 2 trillion tokens of text data in 7 languages — English, Spanish, German, Italian, French, Portuguese and Dutch.
• Stability AI released the pre-training checkpoint to help developers fine-tune the model and full training data details will be provided to enable reproducibility.
• Stable LM 2 Zephyr 1.6B was trained on the Stability AI cluster across 8 nodes with 8 A100 80GBs GPUs for each nodes.
• It can be used commercially or non-commercially with a Stability AI membership.

How to try Stable LM 2

You can access the interactive demo on hugging face.

Here’s an example conversation:

Prompt: What is the distance of the earth to the moon on km?

Response: The distance of the Earth to the Moon is approximately 238,855 kilometers (148,342 miles) as of the date of my last system update, which was September 28, 2021.

Limitations and risks

As with any language model, Stable LM 2 has its limitations.

Due to its size, it may occasionally generate hallucinations or misleading information. Developers should be aware of these risks and implement appropriate safeguards, such as fact-checking and human oversight.

If you’re a developer who considers using this language model for your applications, be aware of these risks and its limitations.

How much does it cost?

If you’re planning on using Stable LM 2 for commercial purpose, you may subscribe to Stability AI’s membership program that costs $20 per month.

You may also contact Stability AI if you want to opt for an Enterprise subscription.

Final Thoughts

Stability AI is really pushing the envelope with the release of this small but powerful language model. Kudos to them for taking a step toward democratizing such technology.

Reducing size while maintaining multi-lingual ability is non-trivial from a technical standpoint and releasing the training checkpoint is definitely a positive step for transparency.

Smaller, more efficient models that can achieve similar performance to larger AI model counterparts are definitely starting to be come more prevalent and are preferable from a “Green AI” perspective.

I can’t wait to see what incredible applications devs will create out of this language model.

‍