Alibaba Cloud, the digital technology and intelligence backbone of Alibaba Group, has announced the open-sourcing of two large language models (LLM): Qwen-72B and Qwen-1.8B.
Containing 72 billion and 1.8 billion parameters, respectively, these versions of the proprietary foundation model Tongyi Qianwen have been made available to the AI model community ModelScope and the collaborative AI platform Hugging Face.
Alibaba Cloud's initiative is another step towards providing multi-modal large language models that can interpret data types beyond textual content to the open-source community. The company has also offered more multimodal LLMs, including Qwen-Audio and Qwen-Audio-Chat, a pre-trained audio comprehension model and its conversationally fine-tuned version, for research and commercial usage.
Jingren Zhou, CTO of Alibaba Cloud, emphasised the importance of establishing an open-source ecosystem to foster the progress of LLM and AI application building. He said, "We aspire to become the most open cloud and make generative AI capabilities accessible to everyone. To achieve that goal, we'll continue to share our cutting-edge technology and facilitate the development of the open-source community together with our partners."
The 72-billion-parameter model, pre-trained on over three trillion tokens, has proven its superior performance over other key open-source models in ten benchmarks. Those benchmarks featured various tasks, from the Massive Multi-task Language Understanding (MMLU) benchmark testing the model’s multi-task accuracy to HumanEval testing code generation capabilities.
The model also demonstrated proficiency in role-playing and language style transfer. This ability allows the LLM to adopt a specific role or persona and generate more contextually relevant responses in line with the adopted persona, a helpful feature, especially for personalised chatbots.
Companies and research institutions are invited to freely use the Qwen-72B model code, model weights, and research documentation. Commercial usage of the models is also free for companies with fewer than 100 million monthly active users. Moreover, Alibaba Cloud has opened access to its 1.8-billion-parameter LLM that can function on the edge, offering a light, cost-effective solution for deploying LLMs that businesses and individuals could find helpful.
As a part of their mission to offer multimodal LLMs, Alibaba Cloud has made available Qwen-Audio and Qwen-Audio-Chat models with enhanced audio understanding abilities. Among other capabilities, Qwen-Audio can comprehend text and audio input in many formats, including human speech, natural sound, and music, while delivering the output as text.
The conversationally refined version, Qwen-Audio-Chat, supports multiple rounds of question-and-answer based on the audio and can perform a range of audio-oriented tasks such as emotion detection and tone analysis in human speech.
Earlier this year, Alibaba Cloud announced the launch of the open-source Large Vision Language Model Qwen-VL and its chat-friendly version, Qwen-VL-Chat. Since their launch in August, these LLMs have registered combined downloads of over 1.5 million on Alibaba Cloud's open-source AI model community ModelScope and Hugging Face. ModelScope, presently the largest AI model community in China, features more than 2.8 million active developers and has reported over 100 million model downloads.