Appraising the validity of a prognosis study

Evaluating the Validity of a Prognosis Study

Are the results valid?

1. Was the sample of patients representative?

The patients groups should be clearly defined and representative of the spectrum of disease found in most practices. Failure to clearly define the patients who entered the study increases the risk that the sample is unrepresentative. To help you decide about the appropriateness of the sample, look for a clear description of which patients were included and excluded from a study. The way the sample was selected should be clearly specified, along with the objective criteria used to diagnose the patients with the disorder.

2. Were the patients sufficiently homogeneous with respect to prognostic factors?

Prognostic factors are characteristics of a particular patient that can be used to more accurately predict the course of a disease. These factors, which can be demographic (age, gender, race, etc.) or disease specific (e.g., stage of a tumor or disease) or comorbid (other conditions existing in the patient at the same time), can also help predict good or bad outcomes.

In comparing the prognosis of the 2 study groups, researchers should consider whether or not the patient’s clinical characteristics are similar. It may be that adjustments have to made based on prognostic factors to get a true picture of the clinical outcome. This may require clinical experience or knowledge of the underlying biology to determine if all relevant factors were considered.

3. Was the follow-up sufficiently complete?

Follow-up should be complete and all patients accounted for at the end of the study. Patients who are lost to follow-up may often suffer the adverse outcome of interest and therefore, if not accounted for, may bias the results of the study. Determining if the number of patients lost to follow up affects the validity depends on the proportion of patients lost and the proportion of patients suffering the adverse outcome.

Patients should be followed until they fully recover or one of the disease outcomes occur. The follow-up should be long enough to develop a valid picture of the extent of the outcome of interest. Follow-up should include at least 80% of participants until the occurrence of a major study end point or to the end of the study.

4. Were objective and unbiased outcome criteria used?

Some outcomes are clearly defined, such as death or full recovery. In between, can exist a wide range of outcomes that may be less clearly defined. Investigators should establish specific criteria that define each possible outcome of the disease and use these same criteria during patient follow-up. Investigators making judgments about the clinical outcomes may have to be “blinded” to the patient characteristics and prognostic factors in order to eliminate possible bias in their observations.

Key issues for Prognosis Studies: