Mistral AI and NVIDIA have jointly announced the release of Mistral NeMo, a 12B parameter language model that pushes the boundaries of multilingual AI capabilities. The model, released under the Apache 2.0 license, offers several key advancements.
Mistral NeMo boasts a context window of up to 128,000 tokens, enabling it to process and understand extensive amounts of information. The model excels in multiple languages, including English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Korean, Arabic, and Hindi.
A notable feature is the introduction of the Tekken tokeniser, which compresses text more efficiently than previous tokenisers, especially for source code and various languages. The model also supports FP8 inference without performance loss due to its quantisation-aware training, enhancing its efficiency.
Mistral NeMo has undergone advanced fine-tuning and alignment, improving its ability to follow instructions, reason, handle multi-turn conversations, and generate code. Comparative benchmarks show Mistral NeMo outperforming recent open-source models like Gemma 2 9B and Llama 3 8B in various tasks.
With its multilingual capabilities, large context window, its release under an open-source license, and availability through multiple platforms, researchers and enterprises alike might be finding (I had to)....