IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

Four things organisations must consider when testing AI

By Contributor
Thu 13 May 2021
FYI, this story is more than a year old

Article by Micro Focus business consultant for enterprise DevOps and hybrid IT management, Matthew Bertram.
 

Machine learning operations (MLOps) is a newly emerging best practice in the enterprise space that is helping data science leaders effectively develop, deploy, and monitor data models. A compound of machine learning (ML) and operations, the market for MLOps solutions is predicted to grow from US$350 million in 2019 to reach almost US$4 billion by 2025. With such rapid growth ahead, it’s crucial that businesses prioritise MLOps innovation now.

Similar to how DevOps emerged from the need to provide a framework for the software development lifecycle, MLOps was developed as a framework for the development of ML systems.

ML development and deployment comprises a complex set of people, processes, and technologies with a lifecycle that needs to be managed, monitored, and optimised to be effective.

Now that businesses have recognised the value of AI and ML, it is important they now focus on extracting the promised value from those ML systems through MLOps.

MLOps in the enterprise space is showing no signs of slowing down. Here are four ways companies can start testing AI more effectively and efficiently:
 

Focus on model deployment

ML mathematical models have a lifecycle that spans from hypothesis to testing, learning, coding, staging, and production. The entire end-to-end deployment process needs to be tracked, monitored, and automated. 

These mathematical models need to be tested and reproduced on new datasets that were not present during the initial development to detect model drift. This is when the conditions or assumptions of the original model no longer apply. Similar to source code and regression tests for software, models need to be version controlled and automatically, continuously tested.
 

Prioritise model security and governance

Attacks against AI and ML models continue to be conducted by bad actors and exposed by leaders in the research community. As MLOps grows in prominence within the IT industry, professionals must incorporate security into the entire AI lifecycle. 

Given ML’s dependency on data, data privacy and ethical considerations must be evaluated and considered frequently. Many AI attacks rely on vulnerabilities that can be easily prevented through regular security reviews and testing.
 

Monitor model performance

It is crucial to monitor the model performance in production because ML is rarely binary and is associated with predictive accuracy. 

Businesses should continuously question how precise the ML model is performing in production on actual data. IT professionals should also measure if performance is decaying or improving over time. 

For example, a model that executes quickly on small amounts of data might find itself struggling with many data points in production, or new data conditions may impact the computational load. It is important to have monitoring systems that measure and record improved model performance and scalability.
 

Automate to scale

Automation through MLOps is critical to scale ML-based production systems. As AI becomes more and more democratised and essential to businesses of all sizes, MLOps will become a crucial requirement for the mass deployment and management of those AI systems. 

During the initial stages of model development, many of the tasks mentioned above are performed by human data scientists or data engineers, using manual tooling and processes. 

While this is acceptable during the initial exploratory development phase, over-reliance on human and manual methods will soon unnecessarily limit production, especially as the number of models grows to the hundreds or thousands.

Currently, MLOps tools are dramatically impacting the IT world, helping increase productivity through automation and intelligence. Decision-makers and IT leaders must consider the role MLOps will play in their business and recognise model performance, security, and scalability as MLOps continues to evolve and grow in the market.

Related stories
Top stories
Story image
Tech job moves
Tech job moves - Bitdefender, Cohesity, Fortinet & MODIFI
We round up all job appointments from June 27-30, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
SOTI
Australian consumers loyal to retailers who deliver speed and visibility
SOTI finds extensive order visibility and speed are the most important factors for turning one-off customers into loyal, long-term buyers.
Story image
Digital
Ivanti puts spotlight on power of employee digital experiences
The report revealed that 49% of employees are frustrated by the tech and tools their organisation provides and 64% believe this impacts morale.
Story image
Artificial Intelligence
Juniper study reveals top AI trends in APAC region
Juniper's research shows an increase in enterprise artificial intelligence adoption over the last 12 months is yielding tangible benefits to organisations.
Story image
Data ownership
Brands must reclaim trust by empowering data ownership
According to Twilio's new State of Personalisation Report 2022, 62% of consumers expect personalisation from brands, and yet only 40% trust brands to use their data responsibly and keep it safe.
Story image
MarTech
Martech experts reveal the “buzz” on personalisation
In the digital age, innovative technology must be leveraged to power an efficient and effective relationship marketing strategy.
Story image
Vendor
Forescout reveals top vulnerabilities impacting OT vendors
Forescout’s Vedere Labs has disclosed OT: ICEFALL, naming 56 vulnerabilities affecting devices from 10 operational technology vendors.
Story image
Migration
SNP unveils next generation of CrystalBridge software platform
Data is a key pillar of every customer-centric organisation, as it relies on agile decisions to become increasingly sustainable and intelligent.
Productivity
Discover the 5 ways your ERP may be letting you down. Is your current system outdated, difficult to manage, and costing you a fortune?
Link image
Story image
Cloudian
Cloudian, Vertica to deliver on-premise data warehouse platform
"We’re enabling our customers to capitalise on a leading object storage platform and maximise the value of their digital assets.”
Story image
Hybrid workforce
How organisations can prepare for a post-pandemic workforce
The so-called 'new normal' office looks different to how it did pre-pandemic, and organisations need to take steps to better manage their post-pandemic workforce. 
Story image
Wiise
Four things wholesale distributors need to consider for FY2023
In a post-pandemic world, there are many things for a distribution business to juggle. ERP solutions company Wiise narrows down what companies should focus on.
Story image
Accounting
Four factors to consider when choosing the right job accounting solution
Progressive job-based businesses can achieve success by strengthening their ability to quantify every cost attributable to the delivery of an outcome for a customer.
Story image
Internet of Things
ManageEngine wins big in IDC MarketScape assessment
ManageEngine's Endpoint Central service has been recognised as a leader by IDC MarketScape in several categories including Internet of Things device deployments and UEM software for SMEs.
Story image
Mergers and Acquisitions
SAS acquires Kamakura to propel risk technology innovation
Underscoring SAS growth in the domain-specific solutions space, the acquisition will enable SAS to greatly enhance the breadth of its risk solutions portfolio. 
Story image
Online shopping
Consumers want speed, visibility in return for brand loyalty
72% of Australian shoppers want complete online order visibility and 63% are loyal to retailers who deliver goods the fastest.
Story image
API
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
Identity and Access Management
Ping Identity named a Leader in Access Management
Ping Identity has been named a leader in the 2022 KuppingerCole Leadership Compass report for Access Management. 
Project management
Discover the 4 crucial factors for choosing the right job-costing solution. Is your team struggling to cost jobs and keep projects running on budget?
Link image
Story image
Cybersecurity
Video: 10 Minute IT Jams - An update from CrowdStrike
Scott Jarkoff joins us today to discuss current trends in the cyber threat landscape, and the reporting work CrowdStrike is doing to prevent further cyber harm.
Supply chain
Discover the 4 critical priorities for wholesale distribution businesses in FY23. Are you worried about how supply chain issues may affect your business in 2023?
Link image
Story image
Data Protection
Five signs your business is ready to move to the cloud
Many organisations are thinking about moving to the cloud. But what are the signs you are ready, and what are the reasons to move?
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Digital Transformation
Discover the 5 signs your business is ready for a cloud-based ERP. Is your business being left behind as more of your competitors switch to the cloud?
Link image
Story image
Cloud Security
Palo Alto Networks bolsters cloud native security offerings
Latest Prisma Cloud platform updates help organisations continuously monitor and secure web applications with maximum flexibility.
Story image
Enterprise Resource Planning / ERP
Five ways your ERP is letting you down and why its time for a change
Wiise explains while moving to a new system may seem daunting, the truth is that legacy systems could be holding your business back.
Story image
Robotics
Evonik relies on Getac F110 tablet to control autonomous robot
The aim of the project is to evaluate the practicality of an automated robotic maintenance and inspection solution in the chemical industry.
Story image
Cybersecurity
Zscaler launches co-located data centres in Canberra and Auckland
The investment will offer public and private sector enterprises greater resilience in support of their zero trust cybersecurity posture.
Story image
Hybrid Cloud
HPE GreenLake advances hybrid cloud experience with new services
"The innovations unveiled today further build on our vision to provide the market with an unmatched platform to spur innovation and drive transformation.”
Story image
Hybrid workforce
Why hybrid working is here to stay and how to ace it
Citrix's new report reveals hybrid workers are more productive and engaged at work than their office and completely remote counterparts.
Story image
Digital Transformation
Google Cloud launches new Digital Accelerator bundles for Aussie SMBs
The new bundles are designed to help Australian small and medium-sized businesses embrace digital transformation and take their businesses online.
Story image
Hybrid Cloud
Advent One acquires Layer 8 Networks, complements hybrid cloud offering
The acquisition comes at a time of surging demand in hybrid cloud, network virtualisation and network security.
Story image
Metaverse
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Story image
Samsung
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.
Story image
Payroll
How New South Wales state departments achieved cloud migration success
State departments in New South Wales are heading to the cloud to achieve better workflow solutions, and one company is paving the way for their success.
Story image
Civil Defence
OutSystems platform chosen as part of ADF contract
"To be included in this project is a reflection of our ability to deliver secure, modern digital outcomes for defence at an incredible pace."
Story image
Sustainability
Vertiv releases updates on ESG initiatives, sets sights on future
Vertiv has released its inaugural environmental, social and governance (ESG) report, the company’s first public report of its ESG activities.
Story image
Compliance
Stock security features inadequate in face of rising risk
"Organisations must proactively find ways of identifying unseen vulnerabilities and should take a diligent, holistic approach to cybersecurity."
Story image
Airwallex
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Story image
Citrix
The best ways to attract young talent during labour shortages
New research from Citrix reveals hybrid working and ventures into the metaverse are top of mind for Gen Z workers.
Story image
Voice recognition
Renesas and Cyberon expand services with voice recognition
“We are honoured to collaborate with Renesas to simplify the development of embedded voice recognition functions."
Story image
Macquarie Data Centres
Macquarie deal to pioneer CO2-cutting data centre tech in Australia
Macquarie Data Centres has signed a multi-year deal with ResetData, an Australian first provider using Submer data centre technology. 
Story image
Robotic Process Automation / RPA
Salesforce announces latest generation of MuleSoft
Salesforce has introduced the next generation of MuleSoft, a unified solution for automation, integration and APIs to automate any workflow.
Story image
Oracle Cloud
Commvault, Oracle to deliver Metallic Data Management as a Service
"We are excited to partner with Commvault and enable our customers to restore and recover their most mission-critical cloud data."