A harm study is research designed to determine if an exposure such as a behavior, treatment, or environmental factor, causes an undesirable outcome. Harm studies may take the form of Cohort Studies and Case Control Studies. To test the validity of harm studies, consider the following issues and questions.
Key Issues for harm studies:
similarity of comparison groups
outcomes and exposures measured same for both groups
follow-up of sufficient length (80% or better)
Cohort Studies
Aside from the exposure of interest, did the exposed and control groups start and finish with the same risk for the outcome?
Were patients similar for prognostic factors that are known to be associated with the outcome (or did statistical adjustment level the playing field)?
The two groups, those exposed to the harm and those not exposed, must begin with the same prognosis. The characteristics of the exposed and non-exposed patients need to be carefully documented and their similarity (except for the exposure) needs to be demonstrated. The choice of comparison groups has a significant influence on the credibility of the study results. The researchers should identify an appropriate control population before making a strong inference about a harmful agent. The two groups should have the same baseline characteristics. If there are differences investigators should use statistical techniques to adjust or correct for differences.
Were the circumstances and methods for detecting the outcome similar?
In cohort studies determination of the outcome is critical. It is important to define the outcome and use objective measures to avoid possible bias. Detection bias may be an issue for these studies, as unblinded researchers may look deeper to detect disease or an outcome.
Was follow-up sufficiently complete?
Patients unavailable for complete follow-up may compromise the validity of the research because often these patients have very different outcomes than those that stayed with the study. This information must be factored into the study results.
Case Control Studies
Did the cases and control group have the same risk (chance) of being exposed in the past?
Were cases and controls similar with respect to the indication or circumstances that would lead to exposure?
The characteristics of the cases and controls need to be carefully documented and their similarity needs to be demonstrated. The choice of comparison groups has a significant influence on the credibility of the study results. The researchers should identify an appropriate control population that would be eligible or likely to have the same exposure as the cases.
Were the circumstances and methods for determining exposure similar for cases and controls?
In a case control study determination of the exposure is critical. The exposure in the two groups should be identified by the same method. The identification should avoid any kind of bias, such as recall bias. Sometimes using objective data, such as medical records, or blinding the interviewer can help eliminate bias.
Source: Guyatt, G. Rennie, D. Meade, MO, Cook, DJ. Users’ Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice, 2nd Edition 2008.
Knowledge Check
A cohort harm study finds a strong association between a chemical exposure and a disease, but the exposed group was older and had more comorbidities than the unexposed group. Which issue most threatens the validity of this study?
A) The researchers did not follow participants long enough to observe the outcome.
B) The study failed to use objective methods to detect the outcome.
C) The exposed and unexposed groups did not start with the same baseline risk for the outcome.