Skip to main content

Comparison Groups

Comparison Groups

Introduction


We populate comparison groups with individuals who closely resemble students in our treatment cohorts, minus the choice to enroll in postsecondary education during the relevant follow-up period.

Individuals in comparison groups are matched to students in treatment cohorts based on a wide range of attributes to maximize resemblance between treatment cohorts and comparison groups and to maximize the accuracy and validity of our comparison group earnings estimates.

In our study, we generate discrete comparison group earnings estimates for the ~8,500 treatment cohorts that we study.

Exact Matching


When identifying comparison group individuals for an entering student in a treatment cohort, we require comparison group candidates to be identical to the entering student in the following ways:

  • Age category (<19, 19-20, 21-22, 23-24, 25-29, 30-34, 35-39)
  • County of high school attended
  • Level of prior postsecondary attainment

Entering students in our treatment cohorts sometimes have prior postsecondary attainment. For example, a bachelor’s degree-seeking student in one of our treatment cohorts might have previously earned an associate’s degree. In such a case, we require individuals in comparison groups to also have a prior associate’s degree.

We limit the number of characteristics of entering students on which we require exact matching to limit the number of students in our treatment cohorts for whom we cannot find exactly matched comparison group individuals.

Composite Matching


In addition, when selecting comparison group individuals for an entering student in a treatment cohort, we seek comparison group individuals who resemble the entering students as much as possible across a range of characteristics, including (but not limited to):

  • Prior earnings
  • Household income
  • Performance on standardized tests of high school math and English
  • Race/ethnicity
  • Gender
  • IEP (individualized education plan) status in high school
  • LEP (limited English proficiency) status in high school
  • Attendance record in high school
  • Pass rates in AP (Advanced Placement) classes in high school

An entrant in a treatment cohort is usually matched to many comparison group members, each of whom is assigned a weight in our cumulative VAE calculations. Matching entrants in our treatment cohorts to many comparison group individuals allows us to improve the quality of our comparison groups and the validity of our comparison group earnings estimates.

Propensity and Prognostic Scoring


A goal of ours in composing a given comparison group is to have individuals in the comparison group with propensity scores that are highly similar to the propensity scores of entering students in the related treatment cohort. 

The propensity score of an individual (whether in a treatment cohort or in a comparison group) can be thought of as a summary measure of the individual’s probability of enrolling in the treatment cohort (whether or not that event occurred).

A related goal of ours in assembling a given comparison group is to have individuals in the comparison group with prognostic scores that are highly similar to the prognostic scores of entrants in our treatment cohorts, at the time of entry. 

The prognostic score of an individual (whether in a treatment cohort or in a comparison group) can be thought of as a summary measure of the individual's earnings potential at the time that the treatment cohort formed.