H2O.ai Announces the Launch of Danube3 Series, Surpassing Apple and Rivaling Microsoft with Latest Small Language Models

H2O.ai, the open-source leader in Generative AI and machine learning, is excited to announce the global release of the H2O-Danube3 series, the latest addition to its suite of small language models. This series, now available on Hugging Face, includes the H2O-Danube3-4B and the compact H2O-Danube3-500M, both designed to push the boundaries of natural language processing (NLP) and make advanced capabilities accessible to a wider audience.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20240718866773/en/

HellaSwag 10-Shot Academic Benchmarks (Graphic: Business Wire)

“We are incredibly excited about the H2O-Danube3 series – a leap forward in making small language models more powerful and accessible. The H2O-Danube3-4B and H2O-Danube3-500M models are designed to push the envelope in terms of performance, outpacing competitors like Apple and rivaling even Microsoft’s offerings. These models are not just high-performing but also economically efficient and easily deployable on edge devices, making them perfect for enterprise and offline applications,” said Sri Ambati, CEO and Founder of H2O.ai.

“With H2O-Danube3, we continue to democratize advanced NLP capabilities, ensuring they are within reach for a wider audience while maintaining sustainability. The versatility of these models spans from enhancing chat applications to supporting research and on-device solutions, truly embodying our mission to bring AI to everyone,” added Sri Ambati.

H2O-Danube3-4B: A New Benchmark in NLP

The H2O-Danube3-4B model, trained on an impressive 6 trillion tokens, has achieved a stellar score of over 80% on the 10-shot HellaSwag benchmark. This performance not only surpasses Apple's OpenELM-3B but also rivals Microsoft's Phi3 4B, setting a new standard in the field.

H2O-Danube3-500M: Compact Yet Powerful

The H2O-Danube3-500M model, trained on 4 trillion tokens, demonstrates remarkable efficiency and versatility. It has achieved the highest scores in 8 out of 12 academic benchmarks when compared to similarly sized models, such as Alibaba's Qwen2. Despite its compact size, the H2O-Danube3-500M is designed to handle a wide range of applications, from chatbots and research to on-device solutions.

Complementing H2O-Danube2 with Advanced Capabilities

The H2O-Danube3 series builds on the foundation laid by the H2O-Danube2 models. The new models are trained on high-quality web data, Wikipedia, academic texts, synthetic texts, and other higher-quality textual data, primarily in English. They have undergone final supervised tuning specifically for chat applications, ensuring they meet diverse user needs.

Key Features:

• High Efficiency: Designed for efficient inference on consumer hardware and edge devices, H2O-Danube3 models can even run fully offline on modern smartphones with H2O AI Personal GPT https://h2o.ai/platform/danube/personal-gpt/
• Open Access: All models are openly available under the Apache 2.0 license on Hugging Face https://huggingface.co/collections/h2oai/h2o-danube3-6687a993641452457854c609
• Competitive Performance: Extensive evaluations show that H2O-Danube3 models achieve highly competitive results across various academic, chat, and fine-tuning benchmarks.
• Use Cases: The models are suitable for a range of applications, including chatbot integration, fine-tuning for specific tasks such as sequence classification, question answering, or token classification, and offline use cases.

Technical Specs:

H2O-Danube3-4B: 3.96 billion trainable parameters, trained with a context length of up to 8,192 tokens.

H2O-Danube3-500M: 514 million trainable parameters, trained with a context length of up to 8,192 tokens.

For more information, please visit www.h2o.ai or H2O Danube3 technical report on arxiv: https://arxiv.org/abs/2407.09276

About H2O.ai

Founded in 2012, H2O.ai is at the forefront of the AI movement to democratize Generative AI. H2O.ai’s open-source Generative AI and Enterprise h2oGPT, combined with Document AI and the award-winning autoML Driverless AI, have transformed more than 20,000 global organizations and over half of the Fortune 500 and household brands, including AT&T, Commonwealth Bank of Australia, PayPal, Chipotle, ADP, Workday, Progressive Insurance, and AES. H2O.ai’s AI for Good program supports nonprofit groups, foundations, and communities in their efforts to advance education, healthcare, and environmental conservation, including identifying areas vulnerable to natural disasters and protecting endangered species.

H2O.ai has a vibrant community of 2 million data scientists worldwide and aims to bring together the world’s top data scientists with customers to co-create GenAI applications that are usable and valuable by everyone. Business users can now leverage the power of LLMs to enhance productivity with enterprise applications.

View source version on businesswire.com: https://www.businesswire.com/news/home/20240718866773/en/