Web Hosting

Free ADS




Our World 2.0






Web Hosting
Sailboat animated gif Sailboat animated gif

How many decimal places do you need?

180px-2spheres intersectingWhen I do a passage plan, especially one which involves one of even two changes of tide, I would count myself lucky to plan to within 1 or 2 nautical miles – relying on adjusting the plan as the passage develops and then pilotage when nearing the destination.

The Wikipedia entry (from which this great graphic was borrowed!) tells us that after the US DOD imposed selective availability program was turned off, the largest error in GPS is usually the unpredictable delay through the ionosphere.

The phase difference error in the normal GPS amounts to between 2 and 3 meters (6 to 10 ft) of ambiguity. Carrier-Phase Enhancement (CPGPS) working to within 1% of perfect transition reduces this error to 3 centimeters (1 inch) of ambiguity.

By eliminating this source of error, CPGPS coupled with DGPS normally realizes between 20 and 30 centimeters (8 to 12 inches) of absolute accuracy.

On a boat with your GPS pitching and yawing to something greater than this then it stands to reason that the software in your GPS is having to try very hard to maintain a reasonably consistent position fix. I presume that the internal arithmetic in the GPS is being conducted with lets say 16 floating point accuracy – but the display of long and lat that it shows you is probably 3 dec places.

So is the GPS Long and Lat to 3 decimal places good enough?

Well, as my corespondent indicated to me below (see comments), because that is equivalent to 111 metres (122 yards or 364 feet)

Each degree at the equator represents 111,319.9 metres or approximately 111 km (= 68.97 miles or 121,391 yards or 364,173 feet ) see the Wikipedia refrence here.

The thickness of your pencil on an ocean passage planning chart could be the same as 1 decimal place (11.1 km) , depending on the scale of the map of course! So, 1 good decimal place is enough for ocean passage planning – but I guess if I was giving my position to the Coastguard I would use as many decimal places as I could lay my hands on!


2 comments to How many decimal places do you need?

Leave a Reply to Whosthis Cancel reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>