[ad_1]
• Falcon Mamba 7B is the world’s top performing open source state-space language model (SSLM), independently verified by Hugging Face
• SSLM has a low memory cost and does not require additional memory to generate arbitrarily long text blocks.
• Falcon Mamba 7B also outperforms traditional transformer architecture models, such as Meta’s Llama 3.1 8B and Mistral’s 7B
• New model reflects Abu Dhabi’s innovation and pioneering spirit in AI research and development
As Falcon’s first SSLM, it differs from all previous Falcon models in that it uses a transformer-based architecture. This new Falcon Mamba 7B model is another example of the groundbreaking research being conducted at the Institute and the groundbreaking tools and products it is making available to the community in an open source format.
Faisal Al Bannai, Secretary General of ATRC and Advisor to the UAE President on Strategic Research and Advanced Technology Affairs, said: “The Falcon Mamba 7B marks TII’s fourth consecutive top AI model win, reinforcing Abu Dhabi’s position as a global hub for AI research and development. This achievement underscores the UAE’s strong commitment to innovation.”
For transformer architecture models, Falcon Mamba 7B outperforms Meta’s Llama 3.1 8B, Llama 3 8B, and Mistral’s 7B on HuggingFace’s newly introduced benchmark. Meanwhile, for other SSLMs, Falcon Mamba 7B beats all other open source models on the old benchmark, and it will become the top model on HuggingFace’s new, stricter benchmark leaderboard.
“The Technology Innovation Institute continues to push the boundaries of technology with its Falcon series of AI models,” said Dr. Najwa Aaraj, CEO of TII. “The Falcon Mamba 7B represents truly groundbreaking work, paving the way for future AI innovations that will augment human capabilities and improve lives.”
State-space models are very effective in understanding complex situations that evolve over time, such as an entire book. This is because SSLM does not require additional memory to digest such a large amount of information.
Transformer-based models, on the other hand, are very efficient at remembering and using information they have processed previously in a sequence. This makes them very good at tasks like content generation, but since they compare every word to every other word, this requires a lot of computational power.
SSLMs can be applied to various fields, such as estimation, prediction, and control tasks. Similar to Transformer architecture models, they also perform well in natural language processing tasks and can be applied to machine translation, text summarization, computer vision, and audio processing.
“As we launch the Falcon Mamba 7B, I am proud of the collaborative ecosystem that TII has fostered to grow it,” said Dr. Hakim Hacid, Acting Principal Investigator at TII’s Interdisciplinary Center for Artificial Intelligence. “This launch represents a significant step forward, inspiring new perspectives and furthering the pursuit of intelligent systems. At TII, we are pushing the boundaries of SSLM and transformer models to inspire further innovation in generative AI.”
Falcon LLM has been downloaded more than 45 million times, demonstrating the remarkable success of these models. Falcon Mamba 7B will be released under the TII Falcon License 2.0, a permissive software license based on Apache 2.0 that includes an Acceptable Use Policy to promote the responsible use of AI. For more information on the new model, visit FalconLLM.TII.ae.
Contact Details
Jennifer Dewan, Senior Director of Communications
Source: Institute of Technology Innovation
Reporter: PR Wire
Editor: PR Wire
Copyright © ANTARA 2024
[ad_2]
Source link
