Tag Archives: quantify

How We Measure Success: Beware Dangers of Metrics Posing as Reality

This do it all scale can tell me my weight, height, body fat percentage, total weight of my fat, and more...if I believe it to be accurate.
This do it all scale can tell me my weight, height, body fat percentage, total weight of my fat, and more…if I believe it to be accurate.

In our hustle and bustle world, it seems there is an increasing pressure to do not just “better” but to do “optimally.” We want to know we could have done no more and be no better than we are. This desire can lead us down many false paths as we attempt to quantify “better” and “best.” How do we measure success?

At 9:48 on April 13, I stepped onto the new scale, height and body fat analyzer at work outside our little convenience store. Then after my lunch run, I decided to see if there was an impact from my run. Since there was no line, I stepped right up. One forty-five minute run resulted in my losing 6.4 lb. of fat! It was such a miracle run, I even gained 0.4 inches in height. The machine even gave me a receipt to prove it!

This machine is a very convenient way to track some basic health stats. However, there is no way I burned 6.4 lb. of fat in a 45 minute run. What this test shows is our need to question the results and measurement error before drawing conclusions. The simplicity of the two measurements claiming to measure the same things seems like a great test, but if results like mine were real, I would be a biggest loser coach on a team that never loses. Alas, life is rarely that simple.

Do not think this is simply about my fat percentage as read by a scale. We make these measurement errors all the time in our desire to have measurable, quantifiable results. With multiple sclerosis, drugs have been approved for more than a decade based on their ability to reduce the number of new lesions seen on MRI’s of patients’ brains. It’s an easy, if expensive, measure which gives researchers a nice quantifiable measurement from which they can claim “drug X is an improvement.” However, the question remains as to whether the lesion test is a good test for the reality of the patients’ multiple sclerosis progression. I and many other MS researchers have come to believe the overall brain volume/shrinkage is probably a better measure of MS damage, but that measurement isn’t enough on its own either to define the damage.

At some point, like after my run, a look in the mirror and noting which belt notch I use is probably the better bet to determine the impact of my run, even if the measurement is less precise than the super scale pretends to be. Similarly, I would never take the results of an MRI showing lesions as the best sole measure of my MS. Some days I will feel awful and tired with new symptoms, and the test results might or might not show why. Neither result changes my reality for all my attempts to quantify the impact of my MS today.

Share