Open Source AI: Falcon, the new champion of language models

0
79
falcon robotics.jpg
falcon robotics.jpg

In the field of open source artificial intelligence, a new contender has emerged that promises to revolutionize the landscape of large-scale language models. This is Falcon, developed by the Technological Innovation Institute (TII) in Abu Dhabi, United Arab Emirates. With its superior performance and a modified Apache license that allows for fine tuning and commercial use, Falcon stands as the champion in this new era of open source AI.

Outstanding performance compared to its competition

Falcon, in its three variants: 1B, 7B and 40B, has outperformed its predecessor, LLaMA from Meta. According to the Institute, FalconLM is the most powerful open source language model to date. The Falcon 40B variant stands out especially, with 40 billion parameters, although it is relatively smaller than LLaMA, which has 65 billion parameters. Faisal Al Bannai, Secretary General of the Advanced Technology Research Council (ATRC), says that the release of Falcon will disrupt access to large-scale language models and allow researchers and entrepreneurs to develop more innovative use cases.

Topping the performance charts

In the leaderboard of open source language models at Hugging Face, two variants of the FalconLM, the Falcon 40B Instruct and the Falcon 40B, lead the ranking, with LLaMA from Meta in third place. Hugging Face evaluates models based on four popular benchmarks: AI2 Reasoning Challenge, HellaSwag, MMLU, and TruthfulQA.

Thorough training and focus on data quality

Although the scientific paper on Falcon has not yet been published, it is known that the Falcon 40B model has been extensively trained using a huge dataset consisting of 1 trillion tokens from the “refined-web” dataset. This dataset underwent a meticulous filtering and deduplication process to ensure its quality and relevance. Importantly, language models are sensitive to the quality of the training data, so considerable effort has gone into building a data pipeline capable of efficiently processing tens of thousands of CPU cores. The pipeline was designed to extract premium content from the web, incorporating extensive filtering and deduplication techniques.

SEE ALSO  The great accessory for your cell phone that you need and don't know: saves time every day and doesn't take up space

Multilingual capabilities and friendly license for commercial use

Falcon models also have multilingual capabilities. In addition to English, they understand German, Spanish and French, and have limited skills in other European languages ​​such as Dutch, Italian, Romanian, Portuguese, Czech, Polish and Swedish. A notable aspect of Falcon is its modified Apache license, which allows fine tuning and commercial use of the models. This is in contrast to LLaMA, whose weights were never open sourced and their commercial use was restricted. Falcon is presented as the first open source language model that goes beyond research limitations.

The push for open source AI and its impact

Open source AI fosters collaboration, transparency, and innovation in the field. Open source models, like Falcon, allow for greater collaboration and knowledge sharing, which can lead to faster progress and innovation. Since the launch of LLaMA, we have seen how these models have given researchers and developers access to powerful language models without the need to invest in proprietary solutions or expensive cloud resources. LLaMA has provided an alternative to closed source models, which some experts have criticized for their lack of transparency and potential bias.

Source: Analytics India Magazine