A study’s external validity is threatened if there may be systematic error in the way its results can be applied to patients outside the precise study set. We should always be concerned about external validity if a study deals with a group of patients who are noticeably different in essential characteristics, relevant to the goal of the study, than the type of patients to whom we ourselves might be interested in applying the results.

Many studies try to evaluate the use of tests to find disease of one sort or another. “Tests” are of course not limited to laboratory studies or X-rays, but for the purposes of this type of research may include a given historical finding, an abnormality on physical examination, a set of “high-yield criteria” or anything we can add to our prior knowledge of patients to further discriminate between those who do or do not have the disease in question.

The importance of sample size is well known in medical research. Use very large samples when comparing two treatments and you will find “true” differences so small as to be unimportant. This month we are going to explore the concept of sample size and discuss ways to read between the lines when analyzing study results.

Some of the major problems in studies have to do with misuse of statistics. Descriptive statistics were originally applied to medical research to control for ways in which the study group, by chance alone, might not exactly reflect the whole universe of patients to whom the results might be applied.

When reading the literature, do you ever wonder who really has the right answers? How can readers come to different conclusions reading the same data? While it is tempting to simply agree with an author’s conclusions, they are often invested in their own work, introducing the potential for bias. Confounding variables are one source of bias that can easily alter the conclusions.

Part 2 in a series - Continuing our discussion on how to understand the literature that we read, we move to a way in which internal validity is threatened: improper classification of results. This can be because of lack of an adequate gold standard, because of biased estimation of results (usually in the absence of blinding), or because of imprecise or irreproducible measurement of results.

Much of the research that we read in medical journals should not be taken at face value, because of a series of errors in study design, analysis (often involving misuse of statistics), and/ or interpretation or extrapolation.  By understanding a few basic concepts we can become adept at interpreting the validity of most of what we read.