- August 14, 2024
- 3 min read
TII Unveils Falcon Mamba 7B: Outperforming LLaMA 3 18B and Other SLMs
In the ever-evolving landscape of artificial intelligence, the Technology Innovation Institute (TII) has once again solidified its position as a global leader with the unveiling of the Falcon Mamba 7B, a groundbreaking addition to its Falcon series of language models. This open-source State Space Language Model (SSLM) has been independently verified by Hugging Face to outshine its competitors, marking a significant milestone in the field of AI research.
A Departure from Tradition
Marking a significant departure from previous Falcon models, which relied on transformer-based architecture, the Falcon Mamba 7B introduces SSLM technology to the Falcon lineup. This innovative approach not only outperforms Meta’s Llama 3.1 8B, Llama 3 8B, and Mistral’s 7B in new benchmarks but also claims the top spot on Hugging Face’s tougher benchmark leaderboard.
Excelling in Complex Tasks
SSLMs excel at processing complex, time-evolving information, making them ideal for tasks like book-length comprehension, estimation, forecasting, and control tasks. The Falcon Mamba 7B demonstrates superior capabilities in Natural Language Processing, machine translation, text summarization, computer vision, and audio processing, with significantly lower memory requirements compared to traditional transformer models.
“The Falcon Mamba 7B marks TII’s fourth consecutive top-ranked AI model, reinforcing Abu Dhabi as a global hub for AI research and development. This achievement highlights the UAE’s unwavering commitment to innovation,” said H.E. Faisal Al Bannai, Secretary General of ATRC and Adviser to the UAE President for Strategic Research and Advanced Technology Affairs.
Embracing Smaller Language Models
In a focused shift to building smaller language models, Hakim Hacid, executive director and acting chief researcher at TII, discussed the institute’s approach in an exclusive interaction earlier this year. “We were asking at some point the question, as to how big should we go? I think now the question is how small we could go by keeping a small model,” said Hacid, emphasizing the need for models that can run on devices and infrastructure with lower computational demands.
With over 45 million downloads of Falcon LLMs to date, the Falcon Mamba 7B continues TII’s tradition of pioneering research and open-source contributions. The model will be released under the TII Falcon License 2.0, a permissive software license based on Apache 2.0, emphasizing the responsible use of AI.
Fostering an Open-Source Culture
TII continues to build on the open-source culture, believing that not everyone will be able to sustain it due to the significant funding required. “You need a lot of funding to sustain open-source, and we believe that not everyone will be able to do it,” said Hacid.
The release of the Falcon Mamba 7B not only showcases TII’s commitment to pushing the boundaries of AI research but also highlights the institute’s dedication to fostering an open-source ecosystem that encourages collaboration and knowledge sharing.
Conclusion
As the world continues to embrace the transformative power of artificial intelligence, the Falcon Mamba 7B stands as a testament to the UAE’s unwavering commitment to innovation and technological advancement. With its superior performance and open-source nature, this groundbreaking model is poised to accelerate AI research and development, paving the way for new breakthroughs and applications that will shape the future of our world.
Stay tuned for more updates on the UBOS homepage as we explore the potential of this remarkable AI model and its impact on various industries.
Related Articles:
- Using Chroma DB with UBOS for RAG AI apps
- Revolutionizing marketing with generative AI
- February product update on UBOS
- Generative AI agents for businesses
- Scaling AI in organizations