What is a propensity score?

Propensity scores: Definition and purpose

The intuition behind the propensity score (PS) is simple: a confounder is only an issue if it is associated with BOTH the exposure and the outcome. If we break the connection between the confounder and the exposure, then this variable is no longer a confounder and will not bias the results. This is done by estimating the probability of the treatment/exposure conditional on the observed confounders. The PS is therefore defined as an individual’s probability of being in the treatment or exposure groups given their observed baseline covariates. The formal notation can be seen below, but essentially, if you ask, “Based on these baseline covariates, how likely is this person to receive treatment/be exposed?”, the PS would be your answer.

Propensity score formal notation

Among cohort subjects with the same propensity score, the distribution of baseline covariates for the treated and untreated group (or exposed and unexposed) is similar. Therefore, once the propensity score has been calculated, it can be used to ‘balance’ the groups of interest. By balancing groups using propensity scores, researchers can account for confounding due to baseline covariates included in the propensity score.

The propensity score distribution is a nice visual representation of the balance in baseline covariates between the two exposure/treatment groups. Plotting a propensity score histogram among those exposed and those unexposed allows the inspection of the overlap between the two exposure groups and the identification of areas with no overlap, which violates the positivity assumption (see the propensity score assumptions section below). Comparing the propensity score distribution before and after applying a propensity score method provides an easy visual technique to assess propensity score balance and compare the balance before and after using the propensity score method. For ideal balance, the propensity score distributions among those exposed and unexposed should be superimposed after applying the propensity score method.

Propensity score distribution

Propensity scores vs regression adjustments

Why use propensity scores when you could just adjust for potential confounders in the regression analysis? Great question! Using regression adjustment allows you to look at the average effect of the treatment/exposure on the individual level (conditional effect). Some propensity score methods allow you to look at the average effect of the treatment/exposure at the population level. In some cases, assuming that the regression model and propensity score have been correctly specified and no unmeasured confounding is present, results from regression adjustment and propensity score analyses will be the same. However, this is not always the case. Therefore, it is important to know which effect you are interested in – individual level or population level effects. If the purpose is to look at the population level effects (similar to a randomized controlled trial), then a propensity score should be used.

We recommend reading the following article for more information (under the section “Propensity score methods versus regression adjustment”)¹

There are additional advantages of propensity score vs regression methods. First, by using the balance diagnostics, it is easier to determine if the propensity score model was correctly specified compared to regression models of the outcome. Second, most propensity score methods are applied at the design stage without reference to the outcome. In contrast, regression adjustment methods are applied at the analysis stage with known outcome and therefore, might be susceptible to bias. Third, unlike regression methods, which are susceptible to overfitting when we have a rare outcome, propensity score methods perform well even with rare outcomes. That’s because instead of including many confounders in the regression model, we’re summarizing all these confounders in one measure, the propensity score. Finally, unlike regression methods, we can explicitly examine the degree of overlap between the treated and untreated groups with propensity score methods. More information can be found in the same section of the article mentioned above ¹.

Exercise 1 – Defining propensity scores

Propensity score assumptions

There are four main assumptions that need to be met for a valid propensity score estimation.

The first is “exchangeability” or “ignorable treatment assignment”, also called the “no unmeasured confounding” assumption. This means all the variables that affect both treatment assignment and the outcome (i.e., potential confounders) have been included. This might be an unrealistic assumption, given that unmeasured confounding is always an issue in observational studies, but it can be addressed in sensitivity analyses to see how robust the estimates are to unmeasured confounding.

The second is the positivity assumption – i.e., all individuals have a nonzero probability of receiving either treatment. In other words, all individuals should have at least some chance of receiving the treatment. This can be tested by examining the propensity score distribution; if we observe an area of no-overlap between those exposed and unexposed, then this violates the positivity assumption (please see the propensity score distribution image above). We can also assess the positivity assumption using descriptives (e.g., min and max), or by using appropriate study design (i.e., making sure the study cohort does not include those who cannot receive treatment, having an absolute contraindication, or those with absolute indication for the treatment).

Areas of no-overlap in PS distribution

The third assumption is stable unit treatment value assumption. This means the potential outcome for one individual is unaffected by treatment assignment of other individuals (also known as “no interference”).

The final assumption is the correct specification of propensity score model. This assumption can be assessed by examining the distribution of baseline covariates between the treated and untreated individuals and using balance diagnostics such as the standardized difference.