Overview

Section learning outcomes:

• Recognize the potential for bias and confounding in observational studies
• Identify the characteristics of experimental and observational designs

Study designs in epidemiology

Epidemiology comprises both the study of the occurrence of disease or health-related states and the study of associations between exposures and health outcomes.¹ Pharmacoepidemiology focuses on the distribution and determinants of drug-related events, including the efficacy and adverse effects of treatments.² Epidemiological studies include trials, observational studies, and meta-analyses. Let’s briefly consider these approaches.

In the 18th century, Scottish naval physician James Lind conducted what became a celebrated clinical trial.³ Interested to test which treatment might alleviate scurvy, Lind arbitrarily assigned the following treatments to sailors suffering from scurvy: cider, elixir vitriol, seawater, vinegar, oranges and lemons, or a mixture of nutmeg, garlic, mustard seed, myrrh, barley water, and tamarind. Each treatment was assigned to two of 12 sailors. After six days of treatment, the two sailors who had consumed oranges and lemons had recovered well, which led to supplying British naval vessels with citrus fruit to avoid scurvy.

James Lind’s test of treatments for scurvy used an experimental design in which treatments were more or less randomly allocated. A well-known observational study examining a cholera outbreak in London was conducted in the mid-19th century by John Snow.⁴ Some residents in an area of London received their water from the Southwark and Vauxhall company, while others in the same area received water from the Lambeth company. The first company’s water supply was from a portion of the Thames contaminated with sewage while the latter company drew its supply from upstream. Snow was able to infer that the Southwark and Vauxhall company’s contaminated water was a source of cholera, because a higher proportion of residents receiving water from that company died from cholera compared to residents served by the Lambeth company.

As these examples attest, trials and observational studies can be powerful tools to examine causes and treatments of disease. But not all studies agree. The use of meta-analysis can help resolve discrepancies apparent in the available evidence. For example, it was unknown whether giving steroids to mothers delivering a premature baby would increase the chance of the baby surviving, because existing studies did not agree on this point.⁵A well-known meta-analysis of existing studies was conducted to resolve this question and found that giving steroids to the mother is highly beneficial to survival of the newborn.

Bias and confounding

As illustrated by the studies described earlier in this section, epidemiologic research may help identify the cause of disease outbreak or a preferred treatment. However, researchers should aim to avoid bias which may arise due to study design or confounding due to differences in groups compared in a study. Bias and confounding can each lead to misleading study findings.

Hormone replacement therapy (HRT) was widely prescribed to post-menopausal women for decades. It was believed that this therapy was cardio-protective, which turned out to be false.³Misleading observational studies of HRT helped encourage widespread prescribing.^3,6 For example, a study published in 1985 suggested that women taking HRT had one-third the risk of coronary heart disease compared to women not taking this therapy.⁷ A randomized trial from the Women’s Health Initiative later indicated that women taking HRT had a 29% higher risk of developing coronary heart disease compared to those taking a placebo.⁸ How could the observational studies and RCT find such different results? 

A likely explanation for the misleading findings from observational studies of HRT is that women who chose to take HRT more generally engaged in health-seeking behaviours than women who did not.^3,6 These behaviours such as exercise, healthy eating or less alcohol consumption likely protected the women from developing coronary heart disease, making it seem that HRT was protective. This type of bias is sometimes referred to as the “healthy user effect” and has the potential to occur in observational studies of preventive therapies.

Confounding may occur in epidemiologic studies when the groups compared in a study differ on an important factor such as age. When such differences are not measured or not accounted for, this can influence the results of a study. This problem is more common in observational studies, but it can occur in randomized trials in cases where the randomization of treatment allocation leads to chance imbalance in patient characteristics in the groups compared. For example, a trial of treatments for diabetes suggested that patients receiving tolbutamide had an additional 4.5% chance of dying over 7 years compared to patients on placebo. Ultimately, it was determined that patients taking tolbutamide did have an increased risk of dying, but the risk was exaggerated because patients assigned to tolbutamide were on average older than patients assigned to placebo.⁴

Comparison of trials and observational studies

Trials fundamentally differ from observational studies in that they are experiments in which the treatment is allocated, usually through a process of randomization, for the purpose of the study, while in observational studies the exposure to treatment is not determined by the researcher.⁹ Clinical trials are considered a gold standard in study design as randomization and other aspects of study design help ensure the validity of the comparison being made. This advantage of an experimental over a non-experimental approach is illustrated by the case of HRT and CHD risk we have already reviewed.

While clinical trials are a gold standard in research evidence, both trials and observational studies have a valuable place in epidemiologic research. It is not always ethical to conduct trials of certain exposures, and observational studies offer certain advantages over trials.⁹ It would not be ethical to recruit patients to a trial to test the harms of smoking a higher number of cigarettes, but exposure to varying levels of smoking has been well-studied using observational designs. Observational studies are typically less costly to carry out, allowing more research questions to be investigated.

Clinical trials are important for investigating the efficacy of drug therapies prior to market approval, but may provide only limited information about drug adverse effects because trials may be limited in size and conducted in narrowly defined populations. Observational studies have a valuable role to play in assessing the association between drugs and adverse effects as larger samples are often available and the effects of drugs can be assessed in the wider population of patients who use the drugs in practice. In this context, observational studies are sometimes said to make use of “real-world data.”

Summary

The remainder of this Module will begin with a discussion of the commonly used measures of disease and association used in epidemiology. From there it will proceed to provide greater detail about basic study designs, starting with trials and meta-analyses and then proceeding to various types of observational studies. As hinted above, key challenges of conducting epidemiologic studies are minimizing bias and confounding. The final portions of this Module will provide an introduction to bias and confounding and how to address them.