Posts Tagged ‘measurement’

So people lie about their height and weight. What’s the problem?

Thursday, May 27th, 2010

Public health  researchers and epidemiologists have little tolerance for inaccurate data.  Their fear is that these inaccuracies could potentially impact the outcomes of research.  They could lead to inaccurate point estimates of effect or somehow produce results whose inaccuracies are impossible to detect.  An example of this potential issue is the ongoing concern regarding problems with self reported height and weight.  This issue illustrates how modern epidemiology analyzes data.

Height and weight are used to compute a person’s body mass index.  BMI , in turn is used to determine whether a person is overweight or obese – a measure that has problems of its own.  Inaccurate height and weight can produce inaccurate measures of BMI.

The easiest and cheapest way to determine height and weight is to ask him or her how tall they are and what they weigh.  No special equipment is needed, no personnel are needed, etc.  What could be easier?

But when researchers compared self reported height and weight to  measured values from data in the National Health and Nutrition Examination Study (NHANES), they found that these self reports were not accurate.  Overall, men said they were taller than they were measured, women self reported lower weights.  Furthermore, the inaccuracies were greater for whites than blacks, black women were the most accurate, Black men were actually more likely to say they were shorter than they were measured and weigh less.

Note that from this data we cannot know the reasons behind these inaccuracies.  People could truly believe they are being accurate or they could be lying.  Who knows?

The problem is that not everyone is inaccurate in the same way.  Overall, the inaccuraciess skew the data in a certain way, but this says nothing about the accuracy of any one individual’s self report.

What should researchers do?  Some suggest that self-reported height/weight data be adjusted to account for these group inaccuracies.  But that makes many researchers uneasy.  How do you know your adjustments would b e appropriate for this particular dataset?

The unknown effects on research outcomes keep researchers (or some of them) up all night.  Are the errors irrelevant?  Are they causing results to appear to be statistically significant when they are really not?    Are  they masking statistically significant associations?  Are they making the point estimates of effect inaccurate?  No one can say at this time.  Also frightening, is this problem going to lead to some skeptic to call for the wholesale rejection of all studies that use self reported height/weight data?   This is not paranoia,  this is a problem that has affected climate change research.

So we watch and we worry and we hedge our findings when we report them.