Melbourne Uni & DARPA to shape AI for scientific decision-making

An international research program will use artificial intelligence to help evaluate the credibility of scientific evidence used to make decisions.

The United States Government's Defence Advanced Research Project Agency (DARPA) will work with the University of Melbourne on the Systematizing Confidence in Open Research and Evidence (SCORE) program.

The University of Melbourne team is the only Australian team that DARPA selected for the research. The university's Collaborative Assessment for Trustworthy Science (repliCATS) project will receive up to US$6.5 million in funding to assess the replicability of thousands of social and behavioural research claims.

''We know that interdisciplinary research can be challenging, however the benefits of drawing upon expertise across our many disciplines are often research outcomes with considerable impact.  I look forward to seeing the insights provided by this exciting and interesting research project,'' comments University of Melbourne acting deputy vice-chancellor (research), Mark Hargreaves.

The project will inform the artificial intelligence component of the SCORE program, and help researchers understand how to recognise credible research.

''This is by far the most ambitious reproducibility project the social and behavioural sciences have seen," says University of Melbourne associate professor Fiona Fidler.

''It will be a defining moment in how we understand the evidence base in the published literature of those fields.''

Fidler will lead the university's team. She is also a reproducibility and open science expert.

Fidler says that the team's experts will work in small groups, locally and virtually, to evaluate published research claims using a proven structured group deliberation approach known as the Investigate, Discuss, Estimate, Aggregate (IDEA) protocol.

''The IDEA protocol is designed to take advantage of diverse points of view, and minimise overconfidence, rather than force premature consensus. We are also interested in the information researchers rely on to make these predictions, and how they collectively reason about the various forms of uncertainty when making these judgements,'' says Fidler.

The repliCATS project is an interdisciplinary research collaboration between the Faculties of Science, Arts and the Melbourne School of Engineering