IT Brief Australia - Technology news for CIOs & IT decision-makers
Story image
AI Chatbots and the Importance of Automated Testing
Tue, 13th Jun 2023

Over recent years, there has been a significant and noticeable rise in the use of chatbots across a wide variety of industries, including banking, healthcare, retail and telecoms, to name but a few. This is revolutionizing how customer service as we know it operates. Generally, customers are happy to have their queries resolved directly through interactions with chatbots. Forrester research highlights that 50% of consumers use chatbots when they have a simple query, and 44% seek them out when they have a question outside of standard business hours. 

According to Call Centre Helper, enhanced customer satisfaction is the main factor driving organizations to integrate artificial intelligence (AI) into their customer service models. These brands also benefit from streamlining their operations and providing faster and improved customer experience (CX). Gartner predicts that by 2026, AI agents/chatbots could save contact centres up to $80 billion in labour costs. 

Chatbots offer several other benefits, including 24/7 availability, instant responses, and the ability to simultaneously handle multiple inquiries. They can also provide more personalized recommendations, automate routine tasks, and significantly reduce waiting times for consumers. 

Critically, chatbots will continue to have a number of limitations and to prevent negative impacts on CX and brand reputation; these must be clearly understood and minimized where possible. For example, while chatbots excel at handling simple and repetitive queries, issues that are more nuanced are likely to require a level of human intervention. This is because if left unmanned, chatbots may misinterpret some queries or fail to understand the relevant context, leading to frustrating interactions for users or even inaccurate responses. 

AI-powered chatbots 

As business leaders understand and become more familiar with AI, they are now using this technology to achieve an array of benefits across their organizations, from product innovation to enhancing data-driven business decisions. A 2022 PwC report revealed that more than 70% of companies were already using or planning to deploy AI within their business operations. 

Rule-based chatbots, which are pre-programmed with scripted answers to FAQs and other common queries, are increasingly being replaced by AI-powered chatbots that leverage natural language processing (NLP) and machine learning (ML) techniques to simulate human-like conversations and interactions. The architecture that underpins these bots generally consists of two main components: natural language understanding (NLU) and natural language generation (NLG). 

To enhance their performance, AI-based chatbots are typically trained on very large datasets of real conversations and are continuously calibrated using techniques such as reinforcement learning. They learn from every user they interact with, and real-time data continuously enhances their understanding and improves their correct response capabilities. 

Some chatbots also employ additional AI-based technologies, such as sentiment analysis to ascertain user emotions or speech recognition to handle voice input. With continuous advancements in ML, NLP, NLU and NLG, chatbots are quickly improving their capabilities and becoming an even more integral part of the modern business landscape. 

The need for testing 

One of the most prominent issues with the use of AI-based chatbots is the inability to control the information it provides users with. AI-powered tools can provide answers to almost any question you may have. But therein lies the main problem — a chatbot is designed to produce content and replicate conversation as if users were conversing with a human. However, this does not mean it is necessarily telling the truth. And that opens up worrying new possibilities for misinformation. In addition, there are significant concerns about data security, phishing, data theft and data manipulation. 

The threats may be numerous and varied, but the solutions for identifying and rectifying them are more straightforward. Many are already familiar with the leadership and management within contact centres, including basic protocols like multi-factor authentication, end-to-end chatbot encryption, and login protocols for chatbots and other AI interfaces. But in the age of AI, true contact centre security must go even further. 

The scope of performance and security testing needed for chatbots is far more extensive than what any organization can achieve through occasional manual testing. Automated security testing provides guardrails and exposes potential weak spots so they can be addressed before they result in a security breach. 

For performance testing, contact centres utilizing increasingly advanced chatbot solutions must commit to regular and automated testing to pinpoint problems as soon as they arise and before they can negatively impact CX and its reputation. 

Tools such as Cyara Botium are designed to help contact centres keep their chatbots running efficiently and correctly. As a comprehensive chatbot testing solution, Botium can perform automated tests for NLP scores, conversation flows, security issues, and overall performance. It’s not the only component in a comprehensive plan for responsible chatbot use, but it’s a crucial one that should not be ignored.


AI can help companies operate more efficiently and address a higher volume of customer issues with less human involvement, leading to significant returns on investment. AI and chatbots haven’t replaced human agents — and likely won’t anytime soon; but they are supporting these agents by crafting a more complete and personalized customer experience, as well as allowing agents to prioritize more complex conversations and tasks. 

Crucially, to enable a smooth experience, contact centres must ensure that their chatbots are performing as they should. That’s why a comprehensive testing solution is essential. Contact centres need to have complete confidence that the chatbots they deploy are running correctly and ultimately avoid causing problems through inaccurate or poor customer interactions.