Elastic moves Jina AI models to faster NVIDIA GPUs

Elastic has begun moving its Jina AI models to NVIDIA RTX Pro 6000 Blackwell Server Edition GPUs, with early tests showing faster processing across several model tasks.

Some Jina API models are already running on the new GPUs, and customers receive the changes without altering their existing setup. The migration covers direct access to Jina models as well as Elastic Inference Service, which provides GPU-backed endpoints integrated with Elasticsearch.

Early results showed the slowest embedding requests running up to 8x faster after the move. Elastic also reported reranking speeds more than 4x faster and document page processing through OCR 3.7x faster.

The work centres on a set of smaller models used in search and AI agent workloads. These include reader, OCR, embedding and ranking models designed to support question answering grounded in content, where latency can rise when several model types are used in sequence.

Mixed workloads

A key part of the testing examined how the newer GPUs handle mixed demand on shared services. In production, large and small requests can compete for the same graphics processing resources, pushing up waiting times for lighter jobs if a larger request takes priority.

This is especially relevant for reranking, where one request may involve a long list of candidate documents while another needs a much smaller, quicker response. In controlled tests, Elastic said smaller reranking requests ran nearly 11x faster even while larger jobs were running on the same system.

The result points to a broader operational issue for companies deploying AI agents and search systems at scale. Smaller models can reduce resource needs, but workload mix is just as important in determining how much GPU capacity a shared service requires.

Elastic compared production traffic before and after the move and carried out controlled tests to isolate cases that are harder to observe in live environments. Latency measurements focused on higher percentiles to capture the slower responses users are more likely to notice.

Model portfolio

Elastic offers Jina AI through two managed services. The Jina API provides direct access to the models, while Elastic Inference Service offers preconfigured endpoints for customers using Elasticsearch.

The OCR models are designed to convert complex documents and websites into structured text, including markdown and HTML for layout and LaTeX for formulas. According to Elastic, the OCR system reads layout, text, tables and structure together in a single pass and supports more than 100 languages.

The embedding models place multilingual and multimodal content into the same embedding space, allowing one query to search across different content types on a single confidence scale. Elastic said the omni embedding models work with text, image, audio and video in more than 100 languages.

The reranking models then rescore search results against the original query so the most relevant matches appear first. Elastic said one of its reranking models, with 600 million parameters, outperformed a 4 billion parameter rival on the STaRK structured data benchmark.

Broader push

The GPU migration is part of a wider effort by Elastic to improve the speed and efficiency of AI applications built around its services. It has also highlighted related work with NVIDIA on cuVS integration as it seeks to refine retrieval and inference performance across its platform.

Elastic said more hosted models from the Jina API and Elastic Inference Service are still being moved to the new GPU architecture. As that process continues, more customers are expected to see the same performance gains as each model is transferred.

One finding stood out from the testing: on NVIDIA RTX Pro 6000 GPUs, small requests stayed fast while large jobs ran alongside them.