Connect finds smaller AI models beat bigger rivals

Connect has published a benchmarking report that found orchestrated small language models outperformed several large language models on enterprise tasks across finance, healthcare and insurance.

The tests examined whether smaller models deployed locally could meet the demands of regulated business environments. Connect compared a baseline small language model, a domain-tuned version of the same model, and an orchestrated ensemble that assigned separate tasks to specialist models.

Across the study, Connect evaluated 19 models in 52 professional domains using sequential editing workflows. Researchers found that, after 20 sequential interactions, frontier models lost or corrupted an average of 25% of document content during extended multi-turn exchanges.

The benchmark used Connect AI Elly, the company's 4-billion-parameter model stack, to measure performance on representative enterprise datasets. The baseline setup, described as a single untuned compact model, scored 72.67% overall.

Domain tuning lifted that score to 81.20%, with strong gains in insurance sentiment analysis and medical question answering. The orchestrated setup scored 85.42%, a 12.75 percentage-point improvement on the baseline and roughly 13 percentage points above the untuned single-model approach.

The orchestrated configuration split workflows into separate functions, including domain routing, entity extraction, summarisation, retrieval and re-ranking, verification, and safety classification. The report argued that this division of labour improved evidence recall as well as financial and clinical summarisation.

It also found that the orchestrated stack outperformed several named foundation models in cross-domain testing, including ChatGPT, Gemini, Amazon Nova, Mixtral-8x22B and Qwen3-14B. According to the findings, it also exceeded Claude in some healthcare and insurance tasks.

The results highlight a growing debate in enterprise AI deployment, particularly in sectors that handle sensitive customer and patient information. Businesses in finance, healthcare and insurance are under pressure to improve automation and retrieval accuracy while maintaining tighter control over how data is processed and where it is stored.

The issue is especially relevant in contact centres, where AI tools are increasingly used to summarise conversations, extract key details and retrieve policy or case information. Errors in those workflows can create operational and compliance problems if systems omit facts, misstate numbers or produce unsupported answers.

Alfredo Gemma, AI Solution Director, Connect, said: "As organisations move beyond experimentation and start deploying AI into operational workflows, there is growing demand for models that are smaller, faster, more cost-effective, and easier to run in controlled environments. The benchmark report reveals that orchestrating multiple domain-specific SLM components improves task accuracy, factual grounding, and summarisation quality by ~13 percentage points overall compared to single-model architectures, outperforming many foundation LLMs."

The report's emphasis on local or sovereign infrastructure reflects a wider trend among companies seeking alternatives to relying entirely on large, general-purpose cloud models. In regulated settings, that can mean using smaller systems tailored to a narrow set of tasks rather than sending broader workloads to one model that handles classification, extraction, summarisation and safety checks at once.

Gemma said the trade-off should be seen as a matter of model selection rather than size alone. "For contact centre leaders and enterprise architects, the choice isn't simply between large and small models; it's about finding the right model for the right task," he said.

He added: "Connect's benchmark report shows that orchestrating specialised models can deliver the domain-specific performance enterprises need, while providing greater control over cost, data and risk. By combining micro models for focused tasks such as entity extraction with domain-tuned SLMs for more complex workflows, organisations can reduce unnecessary cloud processing and reliance on a single vendor. Running this approach on local, sovereign infrastructure also brings greater cost predictability, keeps sensitive data within the organisation's control and enables a configurable verification process to improve transparency and reduce the risk of hallucinations. Ultimately, bigger doesn't automatically mean better. A carefully orchestrated, hybrid approach can give enterprises the performance they need while improving data sovereignty, explainability and return on their AI investment."