Australian company Trellis Data has unveiled a new method aimed at reducing both the cost and carbon emissions of artificial intelligence (AI) systems.
Trellis Data, based in Canberra, claims its Dynamic Depth Decoding (D3) technique allows large language model (LLM) decoders to operate more than three times faster. The company states that D3 utilises a speculative decoding process, reporting an average speed increase of 44% compared to the previous fastest method, establishing it as the speediest decoder available globally.
According to Trellis Data, the implementation of D3 results in text generation that is approximately 3.16 times quicker than conventional methods while maintaining accuracy. This improvement comes as demands for AI continue to escalate, with the World Economic Forum highlighting the doubling of computational power requirements every 100 days to sustain AI's growth worldwide.
Research by Trellis predicts a 68.4% decrease in computational power requirements through the use of D3, which the company argues will significantly cut both the costs and carbon emissions associated with running these AI systems.
Michael Gately, Chief Executive Officer of Trellis Data, commented, "There are three components critical to the success of any AI system: trust, speed and performance. Speed is a key area because it has a direct correlation with cost and environmental impact."
Gately continued, "D3 enables us to address one of the key bottlenecks of speed - the decoder - offering customers a reduction in the cost of running AI servers and a lower carbon footprint."
He further stated, "Trellis Data is at the forefront of the rapidly developing AI field and we're excited to make D3 available on our platform. D3 will provide customers using our speech management and knowledge management capabilities immediate benefits including an enhanced user experience and greater ability to scale."
The D3 technology is applicable across various AI applications such as chatbots, text summarisation, translation, transcription, and content generation. The reduced need for computational resources and corresponding carbon emissions are expected to offer competitive advantages to Trellis Data's customers, especially as LLM usage expands.
Trellis Data employs a diverse range of AI technologies, including generative AI, transcription, translation, and computer vision, all designed to improve efficiency and reduce operational costs. The introduction of D3 aligns with Trellis Data's strategy to support the adoption of new AI technologies while ensuring security and seamless integration with existing business systems.
The company, which has offices in Canberra, Adelaide, Sydney, and Arlington in the United States, is planning further expansion into the Middle East and South East Asia. It supports various sectors, including government, defence, law enforcement, finance, and legal services, by providing tailored AI solutions.
Gately is scheduled to speak at MilCIS 2024 in Canberra, and he is a finalist in the Artificial Intelligence category at the 2024 AmCham Alliance Awards, with the winners to be announced in Sydney.
