New Relic has announced the launch of fhe industry's inaugural Application Performance Monitoring (APM) solution dedicated to AI-powered applications, named New Relic AI Monitoring (AIM). New Relic aims to lead the way in AI observability with AIM in order to offer engineers unprecedented visibility and insights across the AI stack. The goal is to ease the troubleshooting process and optimisation of AI applications with a focus on performance, quality, cost, and regulatory compliance. New Relic's ambition is to empower every developer with observability as a way to advance their organisations' adoption of AI.
The rise of AI in practically every organisation cannot be understated, yet the adoption of Large Language Models (LLMs) is hamstrung by problems such as high costs, inaccuracies, bias, toxicity and hallucinations in responses. These issues inject a vast complexity into organisations' tech stacks, akin to data privacy and security. Trust in AI is hard to attain and easy to lose, with the responsibility falling on organisations to utilise AI responsibly.
New Relic's AIM is now integrated with Amazon Bedrock, a service fully managed by Amazon Web Services (AWS) that facilitates the accessibility of foundational models (FMs) from leading AI firms via an API to construct and scale generative AI applications. AWS customers can use New Relic to acquire superior visibility and insights across the AI stack, easing the troubleshooting process and optimisation of applications for performance, quality, and cost.
New Relic launched AIM in Melbourne as the industry's first end-to-end AI monitoring solution intended to empower engineers to build and operate LLM-powered AI applications with data and insights. These insights allow the optimisation of performance, quality, cost, and compliance, as well as planning for future AI regulations and standards. With more than 50 integrations and features such as LLM response tracing and model comparison, AIM assists teams in building and running LLM-based applications with confidence.
Applications powered by AI introduce an unprecedented level of complexity to organisations. Components of AI such as LLMs and vector databases can often be a black box for engineers, possibly delivering inaccurate and biased results, presenting security issues, and generating large volumes of telemetry data that requires tracking and analysis. Through end-to-end observability of any AI-powered applications, AIM sets out to resolve these issues. Engineers can have a single view to troubleshoot, compare, and optimise different LLM prompts and responses for performance, cost, security, and quality issues, among others. AIM also aims to provide engineers with full visibility on all components of the AI stack alongside services and infrastructure so that they have the necessary data for the responsible use of AI and to validate their compliance with forthcoming AI regulations.
"Just about every organisation is integrating AI applications into their tech stacks to provide better customer experiences and improve efficiency, with the hope of improving their bottom line. The trade-off is that AI creates complexity in their tech stack and must be adopted responsibly with security, quality, compliance, and cost top of mind," said IDC Group Vice President Stephen Elliot. Peter Marelas, New Relic Chief Architect and Head of Technical Specialists APAC, added, "Across the Asia Pacific region, AI has become a key priority with many organisations actively exploring its use in software applications. AIM is an innovative APM solution that provides essential observability for the most popular AI technologies powering today’s AI experiences. Software engineers will, for the first time, have full visibility into the AI stack, enabling them to troubleshoot and deliver safe, secure and reliable AI experiences."
Pinecone Founder and CEO Edo Liberty also discussed how integrating the Pinecone vector database with AIM provides transparency and actionable insights to assist developers in constructing superior search and Generative AI applications within the enterprise. "We have seen incredible demand for vector databases as companies build and deploy AI applications since it is a core part of the AI stack. Now developers can build better search and Generative AI solutions by ensuring relevant and fast responses alongside their AI observability practice." New Relic's AIM is now available in early access to their users worldwide as part of their simplified consumption-based pricing arrangement.