Respondents’ understanding of social survey questions

Example of a question on subjective social status

Abstract

When researchers design survey questions, they must, among other things, ensure the comparability of responses from different respondents. This means that all respondents are expected to understand the question in the same way. Precisely how the respondents understand the survey question and how they formulate the answer to it is the subject of this article. Here, we tackle this kind of problem using the example of a survey question about subjective social status (SSS) used in the International Social Survey Program (ISSP). We used a mixed methods design, specifically a sequential exploratory design, where qualitative analyses of data from cognitive interviews were supplemented with quantitative analyses of survey data. Firstly, we conducted a qualitative study in the form of cognitive interviews using the thinking-aloud technique. We conducted a qualitative analysis using open coding to identify general categories that should capture the content of the interviewees' think-aloud process. Secondly, we used regression analysis to check the correspondence between (objective) socioeconomic status and subjective social status on the data of the ISSP 2019 survey. The results of the regression analyses show that the respondents do take into account their (actual) socio-economic position when self-ranking on the social scale such as education and income. Cognitive interviews show that the respondents differ regarding which dimension of the socio-economic situation they relate to when answering the question on SSS. Furthermore, explanations of self-classification that go beyond the actual socio-economic position are possible as well.