An Evaluation of Biases in Wellbeing Estimates Using Interviewers Versus Online Data Collection in the Global Flourishing Study

Survey mode is a critical consideration in well-being research using the 22 country Global Flourishing Study (GFS). We find marked differences in responses to well-being questions if they are obtained via a telephone interviewer (CATI) or online (CAWI). In fifteen countries in the GFS both survey modes are used. In seven of the most advanced countries, only CAWI is used. Well-being responses are markedly different across survey modes within countries using both modes and compared to countries only using CAWI. On average, CATI tends to produce higher well-being scores. Failure to account for these sampling differences biases results. This is the case in twenty-three prior studies that used the GFS and took no account of survey mode. We examine each of these studies and show the findings differ by survey mode. Combined with the presence of other unobservable country-level confounders, the differences between survey modes substantively weakens the internal validity of cross-country comparisons and random effects meta-analysis conducted with the GFS.