Minister increases pressure for better evaluation

Name: SmartyGrants
Price range: $

Posted on 24 Jun 2024

By Matthew Schulz, journalist, Institute of Community Directors Australia

Gauges Measure shutterstock 148237037 — The federal Labor government continues to push for a greater emphasis on measuring results.

The federal Labor government has renewed its push for better evaluation in the public sector, with the minister in charge calling for widespread adoption of randomised controlled trials, announcing $2.1 million worth of grants for evaluations, and supporting the creation of a new professional network.

Dr Andrew Leigh, the Assistant Minister for Competition, Charities, Treasury and Employment, opened the Australian Evaluation Showcase, hosted at the Australian Centre for Evaluation (ACE), in Canberra on June 17.

In the presence of scores of policy and evaluation wonks who gathered at the capital’s Treasury building, and watched by more than 1000 viewers online, the minister used the anniversary of the ACE to stress the government’s backing of more rigorous evaluation.

The Australian Centre for Evaluation (ACE) was formed with $10 million in funding over four years in the 2023 federal budget and sits within the Treasury department.

Dr Leigh announced a $2.1 million injection of grant funds from the Paul Ramsay Foundation to support up to seven experimental evaluations in the not-for-profit sector. The grants would be worth up to $300,000 each over three years, with winners also supported by the Australian Centre for Evaluation.

Dr George Argyrous – in a later statement – described the grants as “a $2.1 million investment in using experimental evaluation design to increase sector capacity and social impact”.

He said the effort would show that “experimental design can be a cost-effective and efficient way to evaluate areas of social impact”.

Dr Leigh also announced the formation of the Impact Evaluation Practitioners Network, a joint effort of the Australian Centre for Evaluation and the Australian Education Research Organisation, which would foster greater collaboration between government and external impact evaluators.

The network is the latest professional interest group to be established, joining others such as the Social Impact Measurement Network Australia (SIMNA), which highlights best practice, hosts awards, and organises networking events across the country.

ACE teaching presso — The University of Newcastle's Dr Drew Miller presenting findings about teaching research at the event. Picture: UON_research/X

The event, titled Impact evaluation: assessing the effectiveness of Australian public policy, featured the cream of Australia’s evaluators and a keynote by French American economist Esther Duflo, who won a Nobel prize for her groundbreaking use of randomised controlled trials (RCTs) to alleviate poverty.

Professor Duflo – speaking from Paris – joined a string of leading evaluators at the event, and her talk highlighted the many applications for RCTs, as well as their ability to influence public policy.

Other speakers included:

Dr Jenny Donovan from the Australian Education Research Organisation
Professor Sharon Goldfeld from the Murdoch Children’s Research Institute
Professor Julian Elliott from the Future Evidence Foundation
Dr Robyn Mildon from the Centre for Evidence and Implementation
Dr Alex Gyani from the Behavioural Insights Team.

As previously reported by Grants Management Intelligence, Dr Leigh has strongly urged those working for the public service to adopt RCTs in assessing the effectiveness of their work.

RCTs are considered the “gold standard” for evaluation, but many practitioners have doubts about whether they are the best method for evaluating public policy efforts because of their expense. Dr Leigh cited a string of case studies that demonstrated their effectiveness.

“Through the Australian Centre for Evaluation, we are keen to bust the myth that high-quality evaluation is expensive, complicated and time consuming. Some randomised trials can be pricey and continue for decades. But the four studies I cited … were cheap and cheerful, taking less than a year, and costing thousands of dollars, not millions.”

He urged public servants and practitioners to challenge “self-evident truths”, in which false assumptions about education, health, First Nations priorities and many other government activities, meant that evaluations were often “an afterthought rather than built into policy design”.

Dr Leigh also said just 1.5% of evaluations in educational policy and other government programs used randomised trials, but he believed this would change as policy makers adopted a “what works?” approach.

“Just as evidence has increasingly been institutionalised into the training of medical professionals, it is likely that evaluation will become part of the standard toolkit for all Australian public servants. I like to imagine that as public servants apply for promotion to the senior executive service, they will be asked not only ‘tell me about a management challenge you addressed’, but also ‘tell me about the best evaluation you ran.'”