Story image

Lessons from the U.S. election on big data and algorithms

11 Nov 16

The failure to accurately predict the outcome of the elections has caused some backlash against big data and algorithms.

A backlash that is misguided, according to Robert Hetu, research director at Gartner.

“The real issue is failure to build unbiased models that will identify trends that do not fit neatly into our present understanding,” Hetu says.

Hetu says this is one of the most urgent challenges for big data, advanced analytics and algorithms.  

“When speaking with retailers on this subject I focus on two important considerations. The first is that convergence of what we believe to be true and what is actually true is getting smaller,” he explains.

“This is because people, consumers, have more personal control than ever before,” says Hetu.

“They source opinions from the web, social media, groups and associations that in the past were not available to them.:

According to Hetu, for retailers this is critical because the historical view that the merchandising or marketing group holds about consumers is likely growing increasingly out of date.

“Yet well-meaning business people performing these tasks continue to disregard indicators and repeat the same actions,” he explains.

“Before consumers had so many options this was not a huge problem since change happened more slowly. Today if you fail to catch a trend there are tens or hundreds of other companies out there ready to capitalise on the opportunity,” says Hetu.

“While it is difficult to accept, business people must learn a new skill, leveraging analytics to improve their instincts.”

Hetu says the second consideration is closely related to the first, but with an important distinction; go where the data leads.

“I describe this as the KISS that connects big data to decisions,” says Hetu.

“The KISS is about extracting knowledge, testing innovations, developing strategies, and doing all this at high speed,” explains Hetu.

“The KISS is what allows the organisation to safely travel down the path of discovery – going where the data leads – without falling down a rabbit hole,” he says.

Getting back to the election prognosticators, Hetu says there were a few that did identify the trend.

“They were repeatedly laughed at and disregarded. This is the foundation of the problem; organisations must foster environments where new ideas are embraced and safely explored,” he says.

“This is how we will grow the convergence of things we know.”

Is self-service BI living up to the hype?
the explosion of data available to a business and self-service BI tools is transforming how everyone works - but is self-service living up to expectations?
What the people say - Gartner’s November Customers’ Choices
A roundup of the latest Gartner Peer Insight Customers’ Choices from Backup and Recovery to Business Intelligence and Analytics, and more.
How organisations can use AI to generate business insights
DataRobot’s automated machine learning enhanced Precision Marketing’s predictive modelling capabilities.
WA council first to adopt new Datacom tech for local government
The early adopter Shire of Majinup’s initial priority is to use Datascape to help it engage more closely with its community.
Retail-tech firm Insite AI signs deal with Chemist Warehouse
The firm is also raising $2 million in pre-seed funding.
Five secrets – Workday’s 2019 winning formulas
We thoroughly investigate why business software vendor Workday believes 2019 will be their best year yet.
Data protection is key to building customer trust
"New data compliance rules offer an opportunity for businesses to re-evaluate their processes and improve data management and customer loyalty."
Exclusive: Strengths and limitations of the AWS/Cisco partnership
Iguazio CEO Yaron Haviv discusses whether the partnership really is a 'match made in heaven' and what it means for the industry.