2.1. Ethics statement

Ethics approval was obtained by the Ethical Review Committee Inner City faculties of Maastricht University (ERCIC_044_14_07). Participants of the research all provided written consent.

2.2. Sample

We collected data among students at a Dutch Economics and Business school in the first course of the first year. In the Netherlands, curricula of Economics and Business programs typically share several introductory courses, giving rise to classes counting 1000 students or more. The data collection consisted of two parts. The first was held as a mandatory assignment in an introductory course in quantitative methods. This implied that all active students participated in the survey. In this first part, questions about Risk Preference were included. As a result, we have information on Risk Preference of 1056 students. We also have information on the grade obtained in this course of all these students. Risk Preference was measured twice: in week 1 and week 6 of the seven-weeks course. Because we measure Risk Preference vignettes in week 6, we also use data on Risk Preference from week 6. Using the data on Risk Preference from week 1, however, yields very similar results. This implies that Risk Preference remains relatively stable during this education period (the correlation between the measures is 0.555, p<0.0000) and therefore, that reverse causality does not appear to play a role. In the analyses on Risk Preference, we furthermore restrict the sample to those who also answered the questions on Conscientiousness and Emotional Stability in order to show estimations on the same sample for all traits. If we do not restrict the sample, we again find very similar results.

A second survey was held one week after the course ended. Participation in this survey was voluntary. We gave every fifth participant 10 euros and held a lottery among all participants with a 1000 euros prize. In total, 625 students participated. We have information about these students on their Conscientiousness and Emotional Stability.

Table A in S1 File, presents summary statistics on the differences between respondents who participated in the first and second survey and respondents who participated in the first but not the second survey. For the observables we investigate, the groups appear to be comparable although grades are significantly higher in the in-sample and there are slightly more women in the in-sample than in the out of sample group. Note that we cannot compare the level of Conscientiousness or Neuroticism between these two groups since this information was only collected in the second survey.

2.3. Measuring personality

We follow the non-parametric approach developed by [3] and in more detail described by [4]. In their non-parametric approach, they use anchoring vignettes to adjust for respondents’ scale use. In this way, it is possible to recode variables such that respondents’ answers are on the same scale.

We first measure personality of a respondent. For Conscientiousness, we pose the statement: “I am always prepared.” For Emotional Stability, we pose the statement “I get stressed out easily.” For Risk Preference, we ask: “Are you generally willing to take risks, or do you try to avoid risks?” Response categories on all statements and questions range from 1 “I fully disagree” to 7 “I fully agree.” The question we use to measure Risk Preference is taken from the German Socio-Economic Panel and validated by [17]. They show that responses to the survey question predict behavior in incentivized choices under risk. The items we use to measure Conscientiousness and Emotional Stability are taken from standard questionnaires to measure Big Five constructs [18]. In personality psychology, it is common to use more than one question to measure a trait. However, it is important to assess the differences in scale use for personality items separately. Different items may induce different response mechanisms. The wording of items can evoke different interpretations between groups, for example. Grouping items together to reduce measurement error presupposes a common error structure (e.g., all measurement error is independent) of the items. In this paper, we argue the opposite and focus on the item-level to analyze scale use that is specific to the item. As such, we need a vignette for each question we pose. Due to space limitations in the survey, we can therefore only include one item to measure the trait.

Secondly, we ask the same questions, but for fictive persons. For instance, for Conscientiousness, we state the following: “Imagine a surgeon. To what extent does the following statement apply: A surgeon is always prepared.” We have two vignettes for this item of Conscientiousness: “A hairdresser/surgeon is always prepared.” For Emotional Stability, we have two vignettes on one item of the trait: “A hairdresser/surgeon gets stressed out easily.” For Risk Preference, we have three vignettes: “Is a bus driver/firefighter/hairdresser generally willing to take risks, or does (s)he try to avoid risks?”

We then use the answers for the fictive persons as anchoring vignettes for the answers on the subjective question about oneself. The idea is that the way in which subjects report about their own latent trait coincides with the way they report about the latent trait of a “generic” other (e.g. hairdresser). For example, individuals may differ in their interpretation of the trait or answer categories as described in the self-assessment. As the vignette contains the same description of the trait, the vignette presumably captures the same scale use as the self-assessment. In addition, the tendency to agree with statements independent of their content–i.e. acquiescence bias–is captured by both types of assessments. Moreover, the use of vignettes allows us to correct for reference bias. In particular, an individual may evaluate his or her personality in comparison to a reference group (e.g., those who are close like friends, family and colleagues) such that the response portrays an individual’s stance within the reference group. If personality traits in the group are correlated, then someone might report a low score even though they have a high value at the population-level. By introducing a vignette, such errors are corrected.

To do so, we assume a logical order of the vignettes. In the case of Conscientiousness, we assume that surgeons are more conscientious than hairdressers. For the respondents who rated the vignettes in this logical order, we can correct the personality trait by recoding the variable in the following way: C = 1 if y < v1; C = 2 if y = v1; C = 3 if v1 < y < v2; C = 4 if v1 < y = v2; C = 5 if y > v2, where y is the respondents’ self-assessment on the personality trait, v₁ is the response for the same respondent for a fictive hairdresser and v₂ is the response for the fictive surgeon. For Emotional Stability, we assume that a surgeon is emotionally more stable than a hairdresser. For Risk Preference, we assume that a firefighter is more willing to take risks than a bus driver.

In principle, it is an interesting question whether surgeons are observed to be on average more conscientious/emotionally stable. However, for our paper, this is not very important. Crucial for our analyses is that students perceive surgeons to be more conscientious/emotionally stable than hairdressers, or at least they perceive themselves to be on a different level of Conscientiousness/Emotional Stability than those occupations, because the vignettes are mainly about putting everyone’s answers on the same scale. The following percentage of the students indeed do perceive the surgeons to be more conscientious/emotionally stable than the hairdressers: 79.6% of the sample (Conscientiousness) and 56.2% (Emotional Stability). When we allow for a different ordering, these percentages increase to: 94.8% of the sample (Conscientiousness) and 86.0% (Emotional Stability).

Perceptions of the vignettes may differ depending on the occupation of the respondent. E.g., a hairdresser might perceive a hairdresser’s Conscientiousness differently than a surgeon. These influences may be somewhat weaker in the sample, as the respondents are economics students. This firstly means that they are very much alike such that they would all suffer the same “misperception.” Differences between students can then still be interpreted as differences in scale use. Secondly, because we focus on first-year students of Economics and Business, these respondents are not in an occupation yet, besides some small jobs on the side, which are very unlikely to include hairdressing, let alone work as a surgeon. Therefore, we expect that the vast majority of the students will indeed consider service occupations and medical occupations in general.

Respondents do not always follow the intended rank of the vignettes: i.e., either they tie vignettes (e.g., hairdressers are equally conscientious as surgeons) or they misplace them (e.g., surgeons are less conscientious as hairdressers). This leads to the 13 options in Table 1. For Risk Preference, we show two sets of two vignettes in this table: (A) bus driver and firefighter; and (B) hairdresser and firefighter. Alternatively, we can use all three vignettes, but we do not show this in Table 1 since using three vignettes yields 75 ordering options.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The distribution of all options of self-assessment and vignette ratings.

https://doi.org/10.1371/journal.pone.0248629.t001

The bottom rows of the table show how many respondents order vignettes in a logical way. Using the strictest definition of logical order, 80.6% of the respondents ordered the vignettes for the Conscientiousness item “Always prepared” in a logical way. For Risk Preference, both sets of vignettes are ordered logically by almost all respondents (94.4% for set A and 93.4% for set B). For Emotional Stability, we find that fewer respondents logically order their answers. A reason may be that some respondents do not think about a person when answering the questions but about a profession. So, they think for instance, about surgeons working in a stressful environment instead of that a surgeon may be more stress-resistant.

This rescaling technique uses the ordering of the answer relative to the vignettes, which is more objective than the answers on the subjective question. The anchoring corrected personality trait has 5 values: the value 1 if row 1 applies, 2 if row 2 applies, etc.

Besides the most logical ordering of the first five rows, rows 6, 8, 9, and 13 may also be plausible answers since the own value and the vignettes are also separable in these rows. In rows 6 and 8, respondents value the vignettes similarly, but because the own value is above (below) both vignettes, one can still conclude that the own value is lower (higher) than both vignettes. Therefore, for these rows, we can scale the anchoring corrected variable to 1 (5). In rows 9 and 13, the vignettes have been reversed by the respondent, but because the own value is lower (higher) than both vignettes, one can conclude that this is lower (higher) than both vignettes, and therefore we can rescale the answer to 1 (5).

Using this additional insight, we define two samples. The baseline sample contains only respondents with answers 1–5, and the extended sample additionally includes rows 6, 8, 9, and 13. All other answers are less plausible and will be excluded from the analyses.

Tables B-D in S1 File, give summary statistics of the samples for respectively Conscientiousness, Emotional Stability, and Risk Preference.

3.1. Subjective and anchoring vignette corrected measures of personality

We first estimate the following type of functions: in which P_i is the personality trait: Conscientiousness, Emotional Stability, or Risk Preference of individual i. X_i indicates a set of explanatory variables: math major, math entry test score, female, age, parental education, nationality (Dutch (reference group), Belgian, German, other nationality), and study field (Economics and Business (reference group), Fiscal Economics, International Business). The relationship of these variables with personality traits is captured by the vector δ′. The error term is denoted as ε_i.

We run this type of regressions separately for subjective personality traits, and for anchoring vignette corrected personality measures. We do this both for the baseline and the extended sample. We make use of ordered probit models due to the ordinal nature of the dependent variable. The results indicate if relationships between subjective personality and explanatory variables are robust when using the anchoring vignette corrected personality measures.

3.2. Relation between grade and personality

Our main analyses are given by the function: in which G_i is the grade obtained in the quantitative methods course. β′ measures the effect of personality traits on this grade. The relationship of background variables with personality traits is captured by the vector α′. The error term is denoted as μ_i. Similar to the first equation, we estimate this function separately for the three personality traits, which are in turn measured subjectively and anchoring vignette corrected respectively, and we do this both for the baseline and extended sample. We use OLS regressions with fixed effects for the student’s study program. The distribution of the dependent variable Grade is between 1–10 with steps of 0.5-grade point. With 19 different options, we see OLS as best suited in this case. The results indicate if relationships between grades and subjective personality measures are robust when using the anchoring vignette corrected personality measures.