Tech innovators ServiceNow, Hugging Face, and NVIDIA have launched StarCoder2, an open-access family of large language models (LLMs) that provide exceptional improvements in code generation performance, transparency, and cost-effectiveness. One significant feature is the model's ability to train on an impressive 619 programming languages, expanding its accessibility and potential applications.
Developed via the open-source-oriented platform BigCode, the technology is designed to be embedded in enterprise applications, where it could perform specialised tasks such as setting up workflows and generating application source code. Additionally, its advanced capabilities such as code completion, code summarisation, and code snippet retrieval can stimulate innovation and significantly enhance the productivity of developers.
ServiceNow's lead of StarCoder2 development team, and co-lead of BigCode, Harm de Vries credited this achievement to the "combined power of open scientific collaboration and responsible AI practices with an ethical data supply chain". The open-access model substantially benefits developer productivity and offers equal access to the benefits of code generation AI, driving businesses to realise their full potential.
Leandro von Werra, machine learning engineer at Hugging Face and co-lead of BigCode, highlighted the importance of joint efforts in the achievement of "powerful base models that empower the community to build a wide range of applications more efficiently with full data and training transparency".
Johathan Cohen, Vice President of applied research at NVIDIA, pointed out that with each software ecosystem having a unique proprietary programming language, breakthroughs in the development of code LLMs are possible in virtually every industry. NVIDIA's collaboration with the two other firms has resulted in models that are secure, responsibly developed, and promotes broader access to accountable generative AI.
Further developments on StarCoder2 are focused on allowing users to fine-tune the models with industry or organisation-specific data using open-source tools like NVIDIA NeMo or Hugging Face TRL. Organisations can create specialised task-specific abilities, enhancing their businesses' operations.
BigCode represents an open scientific collaboration jointly led by Hugging Face and ServiceNow. Services offered by this group focus on the responsible development of LLMs for code. They also foster transparent supply chain and democratic access to data, while allowing developers flexibility with training sets through the option of opting data out for training.
StarCoder2 was constructed using responsibly sourced data under licence from the digital commons of Software Heritage. Roberto Di Cosmo, Director at Software Heritage applauded the collaboration between ServiceNow, Hugging Face, and NVIDIA as it aligns with Software Heritage' policies for responsible LLM development and portrays a shared commitment to ethical AI development, advancing technology for the greater good.
StarCoder2, similar to its predecessor, will be made available under the BigCode Open RAILM license. This agreement allows users to access and use the technology royalty-free. The supporting code for the models is available on the BigCode projects GitHub page. Developers can experiment with these via downloads, ensuring that AI programming is both democratic and collaborative.
