Changing the measure
August 17, 2025 at 2:49 PM by Dr. Drang
Something Kieran Healy wrote several years ago has stuck with me: If you want to measure change, you can’t change the measure. Kieran attributes that pithy bit of advice to Tom Smith of the General Social Survey, but my Google and Kagi searches always lead to Kieran. Regardless, it’s good to keep in mind.
For example, here’s the past month or so of my resting heart rate according to Apple’s Health app:
Not much change from day to day. But if I look at a graph of last year, I see a big drop from June to July:
What happened? A new medication maybe? No. July 2024 is when I started wearing my watch overnight and either
- the watch’s heuristics say I’m awake and resting when I’m really asleep; or
- because I’m wearing the watch in bed, it’s measuring my heart rate when I’m awake but more rested than I am during the day.
Whichever of these is correct, what’s not correct is that my heart rate suddenly dropped about five beats per minute last year.
I thought about this during the recent dustup over the Bureau of Labor Statistics’ employment figures. One way to make future numbers look better for the Trump administration would be to alter the method used to convert the raw data into the reported figures—to change the measure. This would make comparisons with the old jobs numbers illegitimate, but that wouldn’t stop Trump from doing it.
But coming up with a new method of calculation sounds like a lot of work, and a calculation that gives favorable results now might give unfavorable results a few months from now. Perhaps that’s why E.J. Antoni has floated the idea of suspending the monthly jobs reports. This would introduce a new principle: If you don’t want to measure change, do nothing.