If you're planning on using a wrist-based monitor to track your heart rate while walking, running, or cycling, a group of scientists at Stanford (in partnership with the Swedish School of Sport and Health Services in Stockholm) claim the Apple Watch is the monitor to get, with the smallest margin of error (2%) out of seven tested devices.
The experiment also looked at each device's caloric estimations (or "EE", for energy expenditure). Though the Apple Watch doesn't do poorly in this arena, that doesn't mean much: the lowest margin of error across the pack was 27.4% average, with a whopping 92.6% average error for the Fitbit Surge. In short: Still a long way to go when calculating calories burned effectively on a wrist-worn device.
How was this experiment conducted?
In studies like this, scientists are primarily looking at margins of error when determining what device works "best": In other words, you want a device that regularly reports within a certain margin of error as compared to the control heart rate, or "gold standard."
For this experiment, Stanford used the following for its gold standard:
Because so little testing has been done on wrist-worn devices, there is no "official" standard for such experiments:
As such, the scientists have also proposed a public repository of validated heart monitor data.
To do this first experiment, the scientists identified 45 potential manufacturers, then limited it to eight based on the following criteria:
After excluding the ePulse2, the experiment was left with seven devices.
It is interesting to note that neither Garmin nor Polar's sport-specific wrist trackers were included in this study — we don't know if they were originally considered and then discarded, but it's worth noting given both manufacturers' prior expertise in sport-specific heart tracking.
So what do the heart rate (HR) results mean?
Essentially, after all these tests, the scientists determined that the Apple Watch has the lowest margin of error when it comes to calculating heart rate while walking, running, or biking.
Most of the devices tested came within a median 5% margin of error throughout the tests, with only the Samsung Gear S2 falling outside the range on all activities (5.1% on cycling; a range of 6.5-8.8% on walking; and 6.8% total average).
So the Apple Watch is the best at heart rate for wrist-worn devices, right? According to this study, yes, but its competition is nipping at its heels — a less than 5% margin of error is still quite good when it comes to overall monitoring, so there's no need to throw out your Fitbit Surge if you're otherwise happy with it.
It's also worth noting that this experiment only tested wrist-worn devices in common exercise situations like biking, running, and walking — yoga, weight-lifting, and other wrist-bending activities were excluded, all of which have been known to negatively effect the accuracy of wrist-worn heart monitoring.
What about the caloric (EE) results?
"Calories burned" has always been a bit of a mysterious stat on wrist-worn devices, in part because the calculations behind energy expenditure (or EE) are obscured on a per-device basis. From the study:
As noted above, because there are a lot of variables involved in the calculation of EE — some that require user input, like height, weight, and activity type — it's much harder for any device to give you an accurate estimate. And the study proved it accordingly:
In other words: The Apple Watch may have had the fewest variations in energy expenditure when compared to the other devices in the study, but it still isn't anywhere near the level of accuracy provided by the study's gold standard.
What does this mean for wrist monitors going forward?
For health tech junkies, Stanford's study is actually an incredibly important step forward in getting more reliable data from our devices. Stanford's proposal for a "wearable sensor evaluation framework" alone is a pretty exciting development — if scientists standardize a baseline testing framework and data repository, it allows experiments to be done all over the world with large testing groups, getting us comprehensive data.
Essentially, the more scientific experiments done on wrist-worn devices, the better: More data leads to competition from manufacturers to better their sensors, which gives us (the end-users) even better devices down the line.
And Apple Watch users? For now, you can rest smugly knowing that you'll get a pretty accurate heart rate for most walking, running, and biking activities. (And hope that Apple works on a better system for measuring energy expenditure in the future.)
Master your iPhone in minutes
iMore offers spot-on advice and guidance from our team of experts, with decades of Apple device experience to lean on. Learn more with iMore!
Serenity was formerly the Managing Editor at iMore, and now works for Apple. She's been talking, writing about, and tinkering with Apple products since she was old enough to double-click. In her spare time, she sketches, sings, and in her secret superhero life, plays roller derby. Follow her on Twitter @settern.