We tested 10 GPS devices and smartphone apps to determine whether some devices were more or less accurate than others. In our first report we looked at the distance each device recorded over a 2.5-mile test ride. Today I’ll dissect the elevation data recorded during the same test.
Here’s the raw data:
| Device | Elevation Max | Elevation Min | Elevation Range |
| Asus Android Tablet (Strava) | 1014.44 | 1013.78 | 0.66 |
| iPhone 5 (Strava) | 1015.09 | 1013.78 | 1.31 |
| Garmin Edge 500 | 1077.4 | 1070.9 | 6.5 |
| Magellan Cyclo505 | 1016.2 | 1006.8 | 9.4 |
| Garmin GPSMap 60CSx | 1072.8 | 1047.6 | 25.2 |
| Garmin VIRB Elite | 1043.6 | 1012.1 | 31.5 |
| Garmin Forerunner 405CX | 1044.9 | 1012.8 | 32.1 |
| iPhone 5 (Garmin Fit) | 1039.4 | 998.7 | 40.7 |
| Nokia Lumia (GPS Logger) | 1042 | 1000 | 42 |
| Garmin Fenix2* | 1080.7 | 976.4 | 104.3 |
Since our test was performed on a flat track, the elevation range should be as close to zero as possible. Our best estimate using various online tools is that the track actually has 1 foot of elevation change between its high and low points, and ranges from 1014 to 1015 feet above sea level.
* Note: The Fenix2 experienced a couple elevation “spikes” during the test, including recording one point at an elevation above 1200ft. We removed that point since it occurred at the start of the test, thinking it may have been a reading left over from a previous ride. However, the 1200ft point was actualy the second point recorded (not the first) which seems strange. Either way, the Fenix2 did the worst job collecting raw elevation data during our test.
Hardware
Many high-end GPS units feature a barometric altimeter in an effort to improve elevation accuracy, but truthfully it doesn’t make much difference. As one GPS company rep confirmed, a barometric altimeter is pretty useless unless it’s regularly calibrated. That’s because it uses atmospheric pressure to determine your altitude–but atmospheric pressure changes not just with elevation, but also with the weather. Only one of the top 4 units listed above features a barometric altimeter. The worst-performing unit includes a barometric altimeter as well. And FYI, our test happened a few hours after a massive storm system rolled through.
Services
When it comes to calculating elevation gain and loss over a ride, online services are important tools for improving accuracy. Two of the units in our test–the Garmin Edge 500 and Magellan Cyclo505–produced fairly-accurate raw elevation data, but the data still wasn’t nearly as accurate as the “calculated” elevation gain and loss generated by Strava.
Garmin Connect also attempts to calculate elevation data for rides uploaded to the service, but surprisingly they use a different model (most likely USGS or NASA) that is outdated compared to the one Strava uses. How do we know? Here’s one of the elevation profiles generated by Garmin Connect during our test:
Clearly this is wrong–we were riding around a flat track which obviously didn’t have 30+ feet of climbing per lap. So what gives? My theory is the elevation model being used by Garmin Connect was created by the USGS or NASA before the track was built. This model still shows a hill in the area that was flattened and developed for the track. Will this matter for most, if not all, mountain bike trails? Nope. But it’s still an interesting discovery!
Gross Elevation Gain and Loss
Calculating gross elevation gain and loss from a ride is tricky business because elevation data–whether collected directly via the GPS or calculated using software–often bounces around from point to point and rarely follows a regular, smooth line. So it’s not uncommon for two mountain bikers riding the same loop to wind up with total climb numbers that are hundreds of feet apart.
The plot above was taken from the Magellan Cyclo505, and aside from the first warm-up phase, it actually did pretty well, staying within a range of about 7 feet over 2.5 miles. Still, you can see all the micro-adjustments it’s making, and adding each of these tiny rises and falls can give a false sense of the actual elevation gain/loss during a ride. I ran this data through our own GPSApp.net tool, which calculated about 24 feet of gain/loss over the ride by smoothing the profile out a bit, which isn’t too far off from the estimated gain/loss (1 foot per lap X 10 laps = 10 feet).
Conclusion
So what did we learn from this test? Well, the good news is that dedicated, cycling-specific GPS devices like the Garmin Edge 500 and Magellan Cyclo505 perform better than smaller, less powerful devices when it comes to collecting raw elevation data. Still, that data is far from perfect, and I recommend using online software to validate and smooth elevation data collected using any device.
Your Turn: What GPS myths would you like us to test?
We know that some of these units include an accelerometer so it could be interesting to see how much these help in negating GPS “drift”. It could also be one of the tests done with GPS vs GPS + GLONASS and see if the combo receiver helps as well. Just sit all of the units down in one spot and record for 10 – 30 minutes.
Interesting, I hadn’t considered how the accelerometer might contribute to accuracy. At least one of the units in our test–the VIRB Elite–does include an accelerometer, and glancing at the plot the drift doesn’t look too bad…
In my experience drift is most apparent when the units are stationary. I think the Garmin 510 and above include an accelerometer and they use them to have more accurate speed readings during acceleration and deceleration so it is possible they also use it to assist in removing drift.
You will have to be sure to turn off the “smart” recording feature and record once a second to get some interesting results.
Re: smart recording. We ended up with a mix of recording methods (smart, once a second, once every 10 sec) sort of by accident but in the end decided to use the data for a couple reasons: 1) most GPS users don’t bother changing this setting or don’t know how so it’s pretty good “real world” data and 2) it provided us the opportunity to explain a bit about how GPS units record activities and calculate distances.
For the next test we’ll definitely try to standardize to see how much difference that makes (though I don’t think the smartphone apps we tested allow you to change the polling frequency). Another interesting test might be to use the same unit multiple times, each time trying a different polling method.
Great tests! I’d be interested to see a similar thing done for heart rate monitors, specifically the standard chest straps vs the new transdermal monitors on some smart watches (I think they use LEDs) vs the “bioimpedance” sensor on the new Jawbone UP3.
Garmin (at least the 705) calibrates the barometric altimeter whenever it is started from a known elevation calibration point. I set my calibration point at home, so all my local rides produce calibrated elevation data. As for accuracy, Garmin claims +/- 50 feet with calibration. Precision? i.e. consistency? Any ride starting and ending at the same place should have equal amounts of vertical climb and descent; so, the stats should match. I many, many local rides, the climb and descent numbers are generally close; usually a few tens of feet, sometimes a few hundred. The larger discrepancies usually occur for late afternoon rides, when the temperature falls as the ride ends near or after dusk; not unexpected, since the 705 barometer is not temperature corrected. Significant discrepancy also occurs with noticeable changes in the weather during the ride; e.g. starts sunny and clear, ends cloudy and blustery—also not unexpected. Since I ride the same routes so frequently, I can regard the average data as good, and disregard any large swings day-to-day.
Interesting tests. I would be very interested in a comparison between recorded GPS distance in an extremely hilly trail system with the distance recorded through a traditional wheel sensor and cycle computer. I find that GPS under-reports distance by about 5-10% compared to two different carefully calibrated wheel sensors on two different wheel-sized bikes (26″ and 27.5″). For example, one particular ride in my area that is reported as 12 miles by Strava on a cell phone (Samsung GS5) is reported as about 12.7 miles by means of wheel sensor. I tend to believe the wheel sensor…
The problem with that test setup is we don’t know which one is right. 🙂 That’s the reason we chose a track of known distance rather than assuming one measuring device (say, a wheel sensor) is correct and everything else will be compared to that. That being said, the wheel sensor is *probably* more accurate if it’s calibrated correctly–but it’s not certain.
We plan on doing a second round of track tests to include a wheel sensor and a GPS device featuring GLONASS support. Our original test was to set up determine whether smartphone apps were as accurate as the GPS devices most people use. The next test will expand that to include newer GPS devices and old school tech like wheel sensors.
Actually, you would know which one is right because you have a calibration standard: the track. If you tested your wheel sensor on the track, you could then be sure that it was well-calibrated and then benchmark against GPS distances on a hilly, preferably singletrack course so that your lines were constrained. Of course, you wouldn’t want to jump (freewheeling occurs) or skid, but you have the tools and devices to check this out pretty thoroughly should you desire.
Not a bad idea. Sounds like we need to do a test before we do our test. 🙂