Intergenerational Mobility as a Prediction Problem

In this project we explore a new measurement of mobility based on the predictive power of family circumstances over an individual’s life-time income. We argue that this measure of mobility captures the core concept in a way that is both intuitive and novel. By viewing mobility as a prediction problem, we are able to draw on methods from machine learning. These methods allow for substantial flexibility of functional form and use regularisation to account for ‘over-fit’, allowing us to extract the full predictive content of family characteristics while ensuring we are not mistaking noise for signal. With high-dimensional data, machine learning methods potentially have substantial gains over conventional econometric methods. We compare our measure of mobility to existing measures and argue that this measure captures additional information. Using Norwegian administrative data, we demonstrate interesting heterogeneity across regions of Norway, which we are currently exploring further. As an extension, we are applying these methods to UK survey data, which presents a different set of challenges due to the high number of predictors relative to observations. Preliminary analysis suggests that our approach delivers results on patterns of mobility over time in the UK which contradict the existing literature, due to our different approach to measurement.