Prof. Rodney Ehrlich
School of Public Health
UCT Faculty of Health Sciencesehrlich@cormack.uct.ac.za
LEARNING OUTCOMESBy the end of the session and reading you should have an understanding of:
Note: You may see the Power Point presentation used at the lecture.
This is nicely set out in Hulley pp. 37-39, and is categorised as:
Measurement scales are relevant because the type of scale determines:
Note that continuous variables can be converted into categorical variables. E.g. anaemia present/absent. This categorisation, although frequently done for convenience in research, involves loss of information (and statistical power).
Conceptual variables are expressed in theoretical, general, qualitative, or subjective terms. Our research hypotheses usually start of at this level, for example, “compliance with medication is poorer among patients who lack family support”.
To measure variables, an objective definition is required – this may be a matter of having a readily available validated instrument, establishing consensus or inferring an operational variable from theory (or all three). In this example you would need to have a definition of “compliance with medication” and “family support”.
As part of this process, you will decide on the measurement scale. You may decide to make compliance: “yes/no” (nominal), or “none/ low/moderate/high (ordinal) based on definitions of number of doses taken. For family support, you may do the same: present/absent, or, more likely, use some ordinal scale based on a questionnaire or outsider evaluation.
Another example: “Recovery was faster among those with less pulmonary inflammation at baseline”. Recovery has to be converted into some measurable variable, e.g. “increase in lung function over one year” (continuous scale), as does inflammation, e.g. “neutrophil concentration in broncho-alveolar lavage fluid: (continuous scale).
Precision of a measurement (as per Hulley) is often called reliability, and has the sense of reproducibility or repeatability of a measurement. (Statistically, a measurement which lacks precision or reliability is subject to a lot of random error.)
Hulley distinguishes between accuracy and validity. Accuracy in general refers to approximation to the “truth” as determined by an instrument or test accepted as the “gold standard”.
For a physical or physiological measure, accuracy has the intuitive sense of the instrument being able to get to the real value as it exists in nature.
Hulley reserves the term validity for measures which are based on memory, self-report, subjective, or complex or abstract; for which there is no easily obtainable gold standard even if an objective “truth” exists. Information on medical history and questionnaire responses fall into this category. (“Have you ever had surgery?”, “How many sexual partners have you had”, “Score your pain on this scale”, etc.) . In this sense, validity is a subset of accuracy.
(Note that there is some parallel with the above discussion between a conceptual and operational variable, although the emphasis there was reducing a more abstract concept to a measurable one, rather than comparing one measurement against another).
Both accuracy and validity have the sense of unbiased. (Statistically speaking, if a measurement lacks accuracy or validity, it suffers from systematic error.)
Hulley gives very useful tables setting this out.
This can be set up in a formal pre-study (which you might want to report as a study) or a pilot study (where you generally don’t use the data).
The tests you use depend on the scale of measurement. The tests of precision and accuracy/validity are a little more complex for continuous than for categorical variables.
For simplicity, in this session we will stick to binary (yes/no) categorical variables.
Measures of reliability: percentage concordance, kappa statistic.
Lack of precision of a measurement introduces random error. This leads to a fall in statistical power, and the confidence interval around the estimate widens. This is equivalent to greater uncertainty about the true estimate.
Lack of accuracy or validity introduces systematic error. This leads to a biased estimate which cannot be influenced by repeated measurements or increasing the sample size.
Lack of precision introduces random error, and reduces the likelihood of finding a true difference or association.
2.1 If lack of accuracy or validity affects both groups equally, then in general this reduces the likelihood of finding a true difference.
2.2 If lack of accuracy affects the two groups differently, it may mask or exaggerate any true difference.