Amazon announces AWS Graviton4 & Trainium2 for advanced AI applications

Amazon Web Services (AWS) announced the next generation of chip families, AWS Graviton4 and AWS Trainium2, at the AWS re:Invent event.

These units promise advancements in price performance and energy efficiency, with particular benefits for customer workloads including machine learning (ML) training and generative artificial intelligence (AI) applications. AWS reveals high expectations from Graviton4 and Trainium2, marking them as the latest innovations within its chip design umbrella.

Graviton4 notes an increase of up to 30% in compute performance, possesses 50% more cores, and has 75% more memory bandwidth compared to the incumbent Graviton3 processors.

As for Trainium2, this chip promises four times faster training compared to its predecessor and can be assembled in EC2 UltraClusters as large as 100,000 chips. This innovation makes it possible to train foundation models (FMs) and large language models (LLMs) with significantly lesser time, while concurrently facilitating a double improvement in energy efficiency.

"Silicon underpins every customer workload, making it a crucial area of innovation for AWS," stated David Brown, vice president of Compute and Networking at AWS. He highlighted the increased customer interest in generative AI and asserted that Graviton4 and Trainium2 "help customers train their ML models faster, at a lower cost, and with better energy efficiency."

Graviton4 increases the bar in terms of price performance and energy efficiency, backing a broad range of workloads. AWS already boasts over 150 varying Graviton-powered Amazon EC2 instance types at a global scale.

Widely known companies like Datadog, DirecTV, Discovery, and SAP utilise Graviton-based instances to successfully implement a wide array of workloads such as databases, analytics, web servers, batch processing, and microservices.

Similarly, Trainium2 proves advantageous for high-performance training of FMs and large language models. With the capacity to host up to a staggering trillion parameters, the chip's ability to enhance energy efficiency while reducing costs has been touted as a gamechanger. AWS users can look forward to building models even faster and gaining unprecedented scale and performance.

Datadog, a key AWS user, runs tens of thousands of nodes, making a delicate balance between performance and cost-effectiveness an absolute prerequisite. Principal engineer at Datadog, Laurent Bernaille said, "Integrating Graviton4-based instances into our environment was seamless, and gave us an immediate performance boost out of the box.'

There are similar testimonials from Interactive entertainment company Epic, observability platform Honeycomb, and SAP. All these early adaptors have shared enthusiastic reviews about Graviton4 and Trainium2, eagerly awaiting their general availability to reap their full range of benefits.