To prepare for hiking in the mountainous area of Madeira, I bought a Garmin GPSMAP 67. Same GPS chip as my Fenix 7 and Edge 1040, but a better antenna. Due to the observation of some unexpected odd behavior with respect to the Fenix 7 and Edge 1040 and a disturbing error in the device, I do not trust it. Unless Garmin solves this error I am reluctant to use it. Therefore, I also bought its predecessor the Garmin GPSMAP 66sr as well, second hand. Both devices look the same, but differences exist. On some fora I found indication that this device should be more accurate. So lets put it to the test!
Important notice: during testing I found a serious issue on the GPSMAP 67 on which I reported in another post. Due to this issue the GPSMAP67 is unreliable because it might show your location with a large error. I reported it to Garmin Support but they don’t take it seriously and just refer to their FAQ and state it is ‘by design’. See also this topic on the Garmin forum.
The contestants
The contestants in this test
| GPSMAP 67 | GPSMAP 66sr | |
| GPS Chip | Airoha AG3335M | Broadcom *) |
| Antenna | Quadrifilar helix | Quadrifilar helix |
| Form factor | Hand held | Hand held |
| Satellites | GPS, Glonass, Gallileo, Beidou | GPS, Glonass, Gallileo |
| Firmware version | 8.9 (GPS: 11.02) | 6.10 (GPS: 2.00.13) |
| Graph color | Green | Light blue |
*) Concluded because I found a Broadcom ephemeris URL in the firmware file. Online some reference is made to the Broadcom BCM47755, but it is not explained on which this is based.
Recording mode was interval, one record per second. No clipping to roads. Autostart and Autopause off. Recording only during activity. No power saving (no ‘expedition mode’ etc). Unless stated otherwise, the antenna is pointing up.
About the tests
This post is meant as an addendum for the post in which 4 GPSses were tested. Refer to the test description especially on the definition of accuracy and precision.
Data was retrieved from the .fit and .gpx files as recorded by the device. WGS84 latitude and longitude were converted to the Dutch RDNew map datum since the experiments were conducted in the Netherlands. RDNew expresses coordinates in ‘Easting’ (x) and ‘Northing’ (y) expressed in meter, with respect to some arbitrary reference (in France). This eases further calculation, since we are interested in distances (in m) most of the time.
Accuracy measurement
Coordinate position
An experiment was conducted in which waypoints were logged at a reference point with a known coordinate (Rijksdriehoekscoordinaten kernnetpunt 120324-11 at RDNew coordinate [249021.985, 560371.895]).
The procedure followed was:
- Start GPS, wait until fix, start track logging
- Log a waypoint on the GPSses in quick succession, at the reference point
(Order: GPSMAP 67, GPSMAP 66sr, Fenix 7) - Walk 20 m away from the reference point in one of a few ‘random’ directions
- Walk back to the reference point
- Log next waypoint
The waypoints as logged by all devices appeared to be rounded off:
- The GPSMAP 66sr and GPSMAP 67 log waypoints in a GPX file, with 6 digits accuracy (~1m). Therefore the coordinates were replaced by the coordinate of the track (which are more precise, 8 digits, ~1 cm) at the waypoint datetime.
- In the waypoint list of the Fenix 7 we see the exact waypoint (8 digits) coordinates occur multiple times. Since we assume subsequent waypoints are independent of each other, this implicates that the waypoint coordinates must be rounded off. Since the datetime is not logged to the second but to the minute in the Fenix 7, we cannot use the track coordinates. Therefore in the Fenix 7 graphs we use the waypoint coordinates as-is. It appears there are less waypoints, but this originates from the fact that there are a number of exactly overlapping waypoints.
| GPSMAP 67 | GPSMAP 66sr | Fenix 7 | |
| Waypoints | 30 | 30 | 29 |
| Accuracy | 1.1 m | 1.1 m | 1.8 m |
| Precision | 0.9 m | 0.8 m | 0.8 m |
The GPSMAPs seem to be a bit more accurate than the Fenix 7. The GPSMAPs seem equally accurate, there is no significant difference.
The experiment was repeated a total of four times for the GPSMAP 67. Results are shown in next figure.
| Test22 | Test23 | Test24 | Test34 | |
| Waypoints | 16 | 22 | 16 | 30 |
| Accuracy | 2.5 m | 0.9 m | 1.0 m | 1.2 m |
| Precision | 0.9 m | 0.9 m | 1.0 m | 1.0 m |
We see the precision to be consistently the same. However the accuracy varies in time.
The experiment was repeated at another known coordinate (Rijksdriehoekscoordinaten kernnetpunt 120322-17 at RDNew coordinate [245972.584, 571656.847])
| GPSMAP 67 | GPSMAP 66sr | Fenix 7 | |
| Waypoints | 23 | 23 | 23 |
| Accuracy | 2.1 m | 2.0 m | 2.1 m |
| Precision | 1.1 m | 0.7 m | 0.8 m |
I believe this is one of the most representative tests for logging tracks and waypoints because it is executed under normal hiking conditions. We see that accuracy varies over time between 0.9 and 2.5 m for both GPSMAPs. The precision of the GPSMAP 67 varies between 0.9 and 1.1 m. The precision of the GPSMAP 66sr varies between 0.7 and 0.8 m and is therefore slightly more precise.
Distance measurement – Cycling
The Four Mile running track at Kardinge, Groningen was used to get an impression on accuracy of distance measurement. Two tests were executed:
- Test21 4 laps: GPSMAP67, Edge 1040, Fenix 7, TomTom Adventurer
- Test32 4 laps: GPSMAP66sr, Edge 1040, Fenix 7
- 2017 test 6 laps: TomTom Adventurer
I cycled 4 laps and calculated the distance of each lap. In next graph the center of the circles the average distance. The red circles are from results from the Test32. The radius indicates one standard deviation.
| GPSMAP 67 | GPSMAP 66sr | Fenix 7 | Edge 1040 | Adventurer | |
| Average (m) | 6440.9 (test21) | 6460.3 (test32) | 6453.9 (test21) 6452.6 (test32) | 6452.6 (test21) 6447.7 (test32) | 6453.6 (test21) 6447.7 (2017) |
| Stdev (m) | 3.0 (test 21) | 3.1 (test32) | 4.9 (test21) 2.1 (test32) | 3.2 (test21) 1.3 (test32) | 5.0 (test21) 3.7 (2017) |
| Precision | 0.05% (test21) | 0.05% (test32) | 0.08% (test21) 0.03% (test32) | 0.05% (test21) 0.02% (test32) | 0.08% (test21) 0.06% (2017) |
| Accuracy | 0.06% | 0.36% | 0.26% (test21) 0.24% (test32) | 0.24% (test21) 0.16% (test32) | 0.25% (test21) 0.16% (2017) |
Distance measurement for all devices seems pretty accurate. All within 0.4%. GPSMAP 67 seems to be closest to the 4 Mile distance, GPSMAP 66sr the farthest. However, we do not know the error in the track distance, so this conclusion is not decisive.
To analyse, lets take a few images from the track. Upper row: test32, lower row: test21.
- First of all, the Fenix 7 (blue) and Edge 1040 (red) are very consistent with themselves and with each other. They are also consistent over time. No deviations observed. So for cycling: use one of these GPSses.
- The GPSMAP 67 (green) tends to overshoot corners sometimes or to smooth corners
- The GPSMAP 66sr (light blue) tends to overshoot corners and seems more noisy. This could account for the slightly larger measured distance.
- Note that during the measurement both GPSMAPs were in my breast pocket, which gives it a disadvantage regarding clear sky view (my body is shielding it), whereas the Fenix 7 and Edge 1040 are on my steering rod.
Hiking in the forest
Like stated, I acquired the GPSMAP 67 because I expected it to perform well under harsh hiking conditions in mountainous and forest areas like Madeira. In order to measure quantitatively how GPSses perform, I walked a straight line (deviation +/- 15 cm) while recording the track. I took three devices and attached them to a backpack. The Fenix 7 watch was attached on top of the backpack (center), behind my head. The GPSMAP 67 on my right shoulder, the GPSMAP 66sr on my left shoulder, both vertically. Distance to the body center plane was 18 cm +/- 2 cm. So each device was about 15 cm from my neck and had equal sky view.
In this way I walked a line one way and then back. When walking back, I exchanged the GPSMAPs, so they followed exactly the same line.
First a 900 meter line along the road, with unobstructed sky view was walked, 1 time to and fro. Then two tracks in the forest were walked multiple times (180 m SW-NE 5 times to and fro and 200 m SE-NW 3 times to and fro).
In the track log, a fit was made throught all the measured point from the three devices. This fit was regarded as the acual location. The distance to the fitted line was calculated for each device. Distance to the line (ε cos(α)) is plotted against the distance along the line.
The experiment was repeated in a dark production forest in which the trees are quite close to each other and in straight lines, enabling to walk straight lines.
Note that the y-scale for the road measurement is smaller. The red line shows the expected location, indicating the off center placement of the GPSMAPs. Next table shows the average error |ε| (accuracy):
| GPSMAP 67 | GPSMAP 66sr | Fenix 7 | |
| Unobstructed sky view | 1.3 m | 1.1 m | 0.5 m |
| Forest | 1.5 m | 1.8 m | 3.8 m *) |
| Dark production forest | 1.8 m | 2.1 m | 3.1 m |
*) During the hike in the first forest, the Fenix 7 might have been not oriented face up; it might have been covered by the top of the backpack. It may account for the relatively high value. In the second forest we made sure its face was up.
Finally I walked a rectangle in the dark forest, between the trees, with measured sides, 10 times counter clockwise (bright color), 5 times clockwise (dark color). The circumference was 116.2 +/- 0.2 m, measured with a tape measure. GPSMAP 67 on right shoulder (outside on CCW, inside on CW), GPSMAP 66sr on left shoulder (inside on CCW, outside on CW), Fenix 7 behind head.
| GPSMAP 67 | GPSMAP 66sr | Fenix 7 | |
| Average lap | 114.6 m | 121.2 m | 110.3 m |
| Error | -1.4% | +4.3% | -5.1% |
The [0 ,0] origin is the average of all coordinates recorded. The purple rectangle is the measured rectangle placed in the orientation of the trees (longest side). Unfortunately I am not quite sure if the angles are exactly 90 degrees. I estimated them ‘by eye’. The GPSMAP 67 appears to be very precise. Based on the weighted average distance it also the most accurate.
In order to conclude we should actually have more measurements. Given this measurements we conclude following:
When hiking under unobstructed sky conditions, the Fenix 7 appears to be more accurate. Average error is 0.5-0.7 m. The GPSMAPs show an error of 1.1-1.3 m. There is not a significant difference between both GPSMAPs. The GPSMAP 66sr however seems to be more precise.
Going into the forest the precision as well as the accuracy deteriorate. Now the GPSMAPs show their value: the accuracy deterioration is limited: from 1.1-1.3 m to 1.5-1.8 m. The accuracy deterioration from the Fenix 7 is far bigger: from 0.5-0.7 m to 2.4-3.8 m.
In the dark forest the GPSMAPs slightly deteriorate more. When looking at the rectangle, we see the GPSMAPs doing quite well. The Fenix shows rounding off.
GPSMAP 67 appears to be slightly more precise and accurate than the GPSMAP 66sr.
Mountainbiking in the forest
I took four GPSses to the forest.
| GPSMAP 67 | GPSMAP 66sr | Fenix 7 | Edge 1040 | |
| Location | Right back pocket | Left back pocket | Left wrist | Steering rod |
| Graph color | Green | Light blue | Blue | Red |
Next picture shows three typical situations
Left: good accordance between the GPSses
Middle: typical overshoot of the GPSMAP 66sr (light blue)
Right: typical overshoot of the GPSMAP 66sr (light blue) and smoothing of the GPSMAP 67 (green)
So we conclude that the Edge 1040 and Fenix 7 perform more accurate than the GPSMAPs for cycling.
Hiking the urban jungle
To do
Precision measurement
Stationary placement
We left the GPSMAP devices for a longer period outside, upright, with unobstructed sky view. They were placed and fixed on a plank which was oriented east-west. Distance between antennas was 78 +/- 1 cm. The assumption was that they would continuously log the coordinate with a certain error. By leaving the devices at the coordinate we would get an impression of the average error and that we could get a close match to the actual coordinate by averaging the logged coordinates.
Next image shows the result of one of the measurements. Each logged coordinate (every second) is plotted as a dot. Dark circle indicates the circle for which 50% of the measurements fall within (CEP). The red circle indicates the circle for which 95% of the measurements fall within.
| GPSMAP 67 | GPSMAP 66sr | |
| Placement | east | west |
| Measurements | 52884 | 52902 |
| Duration | 14h 41m 27s | 14h 41m 38s |
| Average distance | 0.92 m | 1.2 m |
| Maximum distance | 2.9 m | 3.6 m |
| CEP | 0.83 m | 1.2 m |
| CEP 95% | 1.9 m | 2.1 m |
| Average Coordinate | [244757.240, 559288.223] | [244755.860, 559288.596] |
The pattern of the GPSMAP 67 is the pattern we might expected, since the coordinate seems to be logged with some error. However, we observe that the GPSMAP 66sr shows a more irregular pattern with clustering instead of nicely distributed. That is also visible if we make an histogram on the distance.
I would have expected a Gaussian distribution, and therefore the histogram to have its maximum at 0 distance. Apparently I am mistaken.
Therefore, the experiment was repeated several times. Next graph shows the average error and the 95% radius as function of the measurement time, together with an exponential fit.
We see that the GPSMAP 67 is slightly more precise, though not significant.
Next plot shows the average coordinate. The origin [x,y]=[0,0] is the average of all the average coordinates shown. The red dots then are the expected coordinates, 78 +/- 1 cm apart from each other. Average coordinates are shown as circles, the radius is the expected error. It is the average measurement error devided by √N/5, where N is the number of measurements (seconds) and 5 is 5 seconds, the time after which a subsequent GPS coordinate measurement may be regarded as independent from the previous measurement.
Having presented this, we have to place a note on the GPSMAP67. We observed during these measurements odd behavior as is shown in next picture. The logged coordinate occasionally tends to deviate extremely. This effect is more extreme and repeatable if the device does not have clear sky view, as we reported in this post. In the measurements above, these extremities have been left out.
It is quite hard to derive conclusions from these measurements. But let’s try it…
First of all this is not a very representative measurement for real life situation. We use GPSses for tracking our activities, not for accurately measuring coordinates, though in this article the GPSMAP 66sr has been proposed to do cm accurate coordinate measurements.
The GPSMAP 67 seems a bit more precise in logging coordinates in a stationary situation. Average error is 1.1 m for the GPSMAP 67, 1.5 for the GPSMAP 66sr at 24 hours of measuring.
The accuracy of the average coordinate is not as high as we expect it under the assumption that it takes 5 seconds for a measurement to be independent from the previous measurement. This can be explained by the fact that GPSses apply all kind of dark magic, like smoothing, averaging, Kalman filtering, etc to get more accurate readings under movement. Obviously, the GPSMAP 66sr does things differently than the GPSMAP 67. Stated otherwise: GPSses like the GPSMAPs are not meant for stationary measurements but require movement to do its job.
Anyway, regarding accuracy in stationary situation, the GPSMAP 67 seems to be more accurate, based on the coordinate graph. However, the GPSMAP 67 sometimes shows extreme drift, which makes it totally unreliable in this experiment. As is shown in this post it sometimes leaves a persistent error.
Stationary placement, vertical vs. horizontal
Both devices were tested vertically and horizontally, for each orientation 1.5 hours. When horizontally device was put on a block, antenna put over the edge, so the antenna was clear of the ground and clear around. The antenna was pointing to the north.
| GPSMAP 67 Vertically | GPSMAP 67 Horizontally | GPSMAP 66sr Vertically | GPSMAP 66sr Horizontally | |
| Duration | 1h 31m 53s | 1h 33m 39s | 1h 31m 46s | 1h 33m 29s |
| Average distance | 0.64 m | 1.22 m | 0.86 m | 0.75 m |
| Max distance | 2.34 m | 3.33 m | 1.88 m | 2.58 m |
| CPE | 0.57 m | 1.15 m | 0.73 m | 0.69 m |
| Average Coordinate | [244756.8, 559287.4] | [244756.3, 559289.1] | [244757.6, 559289.1] | [244755.8, 559289.5] |
Looking at the GPSMAP 67 we see the earlier observed behavior that when the device is placed horizontally, the error increases, especially in the direction along the antenna. In the direction perpendicular on the antenna there is not much difference.
Looking at the GPSMAP 66sr, we see the opposite. However, the measurement in vertical orientation deviates from the measurement above (test28; here the average distance was 0.54 m against 0.86 m in this exeriment). There must be a special cause. If we take the test28 measurements in account we also see the same behavior with the GPSMAP66sr: in horizontal position the error increases.
Conclusion
- Based on the logging of waypoints at a known reference we found the GPSMAPs to be equally accurate, though the precision of the GPSMAP 66sr seemed a little bit better. They perform more accurately than the Fenix 7.
- For cycling the Edge 1040 and Fenix 7 are unsurpassed regarding precision. They are very consistent with themselves as well as with each other and over time. GPSMAPs tend to show overshoot or smoothing making them less consistent for cycling.
- When hiking under unobstructed sky conditions the Fenix 7 performs with better accuracy than the GPSMAPs. However, when conditions worsen, like under the trees, the GPSMAPs show their true value regarding accuracy and precision. So for hiking in nature under outdoor conditions, I would advise a GPSMAP.
- When placed stationary the GPSMAP 67 seems more accurate and precise than the GPSMAP 66sr if it weren’t for the extreme coordinate drifts that occasionally happen on the GPSMAP 67.
- The coordinate drift issue as reported in the other post makes the GPSMAP unreliable under situations where there is no movement, for example Geocaching, unless Garmin solves this issue.
Bottomline: there is not a significant difference between the GPSMAP 66sr and GPSMAP 67.
Further reading
- US Government: URE or User Range Error is <2.0 m for GPS with 95% probability. In practice, it is better that this. URE is not User Accuracy. The latter also takes environmental factors into account.