Governments have many programs that seek to improve disadvantage and ameliorate poverty. But how successful are they at enabling people, households and communities to leave behind their disadvantage and break the cycle of intergenerational disadvantage?

Governments, researchers and NGOs are seeking to better understand what the most powerful interventions are. What will materially improve the future prospects for children born to parents of a certain income, wealth, location or social group?

To understand the net impact of the interventions is a major organisational, evidence-gathering, and data research challenge, especially given the task of measuring long-run impacts.

Considering social mobility through place

One of the leading academic experts in social mobility, Raj Chetty, re-evaluated jointly with his colleagues the effects of a housing experiment implemented in three large US cities in the 1990s, namely Chicago, Los Angeles and New York.

The Moving to Opportunity experiment randomly allocated housing vouchers to a selected group of eligible individuals to move from high poverty neighbourhoods into low poverty neighbourhoods. The first evaluation (Katz et al, 2001) did not find any sizable positive difference between earnings of those who got the vouchers (the treatment group) and those who did not (the control group).

The earlier the intervention, the more value for money

A few years down the line, they found the experimental vouchers had indeed been successful, just not among eligible adults, but their children instead. Compared to the control group, children in families who moved to more prosperous areas ended up earning a 31% higher income on average, were 27% more likely to attend university and 30% less likely to become single parents than others in their age group.

Furthermore, the younger the child was when they moved out of a poor neighbourhood, the larger the income gain. In fact, moving after age 21 had virtually no impact on future adult earnings and also did not affect parental income.

Early childhood interventions are the key to social mobility

This findings suggests early life circumstances, rather than local labour market opportunities, are key to fostering intergenerational mobility. This is backed up by evidence-based research led by James Heckman, a Nobel laureate economist, showing that early childhood interventions yield extremely high economic and social returns in terms of adult wages (through higher tax contributions), reduced crime, decreased welfare dependency and crime incidence, as well as improved health.  

The power of states, regions and councils  

Australia has a very different setting than the US, where there are extremely large intergenerational mobility variations even within a city. But the Chetty work demonstrates the benchmark research approach needed to properly evaluate how to improve social mobility.

A local database with geographical localisation and measures of intergenerational mobility created by Chetty for the Equal Opportunity Project has enabled US policy makers to detect and prioritise the most problematic areas. The data exposes areas with a high density of families who can benefit most from mobility, which means they can be identified and targeted to obtain the highest returns on investment in developing human capital.  

The need for more accurate data to identify and implement cost-effective policies

This is a classic big data challenge, which in Australia is now being led by the Australian Research Council Life Course Centre of Excellence, working together with federal and state agencies to facilitate the access and linkage of administrative data from multilateral governmental bodies. The multifaceted nature of social mobility necessarily demands a co-ordinated commitment from all the relevant agencies involved to provide efficient data access. Much of this data and insight will need to come from local authorities and service providers to ensure the specificity needed to ensure programs are tailored to local needs.

A call for evidence-based policy interventions in Australia

Once data is available, the second key challenge is designing robust evidence-based policy evaluations, relying on randomised controlled trials or quasi-experiments. Chetty’s MTO experiment was able to measure with confidence the impact of the housing vouchers because they were allocated randomly to individuals who were similar in treatment and control groups. For obvious reasons, such evaluations should not be performed by government agencies but pursued and designed by highly-regarded independent researchers, working together with the government. Unfortunately, social program evaluations in Australia are rarely randomised and independently evaluated.

Randomisation allows for accurate measurement of outcomes

Policy interventions will be misleading if individuals eligible for say, social housing, choose to apply to be reallocated to a higher income neighbourhood instead of being randomly assigned.

If, for example, we observe that children of families who choose to be reallocated have higher earnings than those who do not sign up in the future, we cannot tell whether the program worked or those who were aware and applied for social housing are just different and more motivated than those who did not. Thus, we are unable to disentangle the effects of the program vs family characteristics if there is a selection of families with better prior characteristics who apply for reallocation.

A randomised pilot experiment with a large enough sample allows us to measure and compare the relative cost-effectiveness of different policies to choose from, when budget allocations need to be made.

Learning from mistakes

Many well-intended interventions can generate unintended consequences that we need to consider and understand. Several examples come from medicine, the discipline with the longest history of using RCTs. For instance, certain policies aimed at reducing teenage childbearing had the complete opposite effect. This strongly suggests we need to learn from experimental pilots, before rolling out universal policies.

And while some may argue that randomised trials benefit some who are not in need of help, the bigger benefit comes from being able to target programs that have a proven impact, rather than investing in large scale programs which have little benefit to the actual groups in most need.

The alternative: quasi-experimental evidence

When RCTs are not feasible, there are still certain policy designs that allow for evidence-based evaluation without explicitly generating control and experimental groups. For instance, program assignment lotteries or income eligibility thresholds allocate individuals who are arguably relatively similar into quasi-treatment and control groups, due to reasons beyond their control. While such program evaluations have certain caveats, they at least allow for second-best evaluations that are better than no evaluation at all.

Policy and research should go hand in hand

Given the complexity of evidence-based design of interventions and evaluations, it is obvious researchers and policy makers need to be working closely together to exchange evidence-based policy ideas and design policies that can be evaluated by independent parties. The ARC Linkage grants and the newly inaugurated Sydney Policy Lab at the University of Sydney are steps towards the right direction.

To drive such a broad based concept like social mobility needs a strongly coordinated and focused program of agencies, universities, NGOs and service providers, supported by government data and policy diagnosis.

References

Chetty, Raj, Hendren, Nathaniel and Katz, Lawrence F (2016). The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment’, American Economic Review 106(4): 855–902.

Katz, Lawrence F., Jerey B. Liebman, and Jerey R. Kling (2001). Moving to Opportunity in Boston: Early Results of a Randomized Mobility Experiment. Quarterly Journal of Economics, 116 (2): 607-654.