Eye, Robot: A Generative AI Approach to Measuring Subjective Assessments from Street View Images at Scale
We introduce a new method for measuring subjective perceptions at scale using innovations in both crowd-sourced image data and transformer-based computer vision technology. We aim to uncover the relationships between perceived neighbourhood conditions and political outcomes, potentially offering new insights into how urban environments shape politics. Our approach relies on a novel dataset of 360-degree street-level images from Mapillary, a non-proprietary crowd-sourced alternative to Google Street View. We move beyond conventional image-as-data methods like object detection, using OpenAI’s GPT to label images for subjective concepts like disorder, safety, and perceived wealth from unlabelled images of neighbourhoods. We compare GPT’s labels to a sample of crowd-sourced labels, as well as objective data at the image-level (using object detection models) and geographic data on crime reports, 911 calls and socioeconomic characteristics. We plan to examine the association between these subjective assessments of neighbourhood characteristics and political outcomes including voter turnout and voting behaviour on a relevant ballot initiative intended to reduce blight. In doing so, we home to demonstrate 1) the value of incorporating measurements of subjective perceptions over and above that of objective physical, social, and economic features and 2) how using GPT-derived assessments can be used to proxy subjective perceptions, on average and within subgroups, making their measurement more feasible. In an initial pilot presented here, we compare GPT-derived measures against human perceptions derived from crowd-sourced labels, for a small subset of images. Correlations with objective attributes such as image features, 911 calls, and racial composition are explored as well. We plan to then generate systematic measures for the entire city of Detroit at scale, and ultimately expand to other cities.
Date: 3 May 2024, 13:00 (Friday, 2nd week, Trinity 2024)
Venue: Manor Road Building, Manor Road OX1 3UQ
Venue Details: Seminar Room A
Speaker: Melissa Sands (LSE)
Organising department: Department of Politics and International Relations (DPIR)
Part of: Politics Research Colloquium
Booking required?: Not required
Audience: Members of the University only
Editors: Holly Omand, Daniel Burton