Deciding how many decimal places to include when reporting latitude and longitude

When giving coordinates or any other measurement, a good rule of thumb is that the number of decimal places included (if any) should be consistent with the precision used to collect the measurement or with the intended application. For example, if you visually estimate the distance to a building from a road you might record the distance in metres, but if you use a measuring tape recording the distance in centimetres would be appropriate. Giving a higher precision (more decimal places) than necessary can be misleading as to the accuracy of the measurement.

Since lat/long is an angular unit, it is not obvious what changes to the precision actually means on the earths surface. For example, what would the distance be between coordinates given to 4 and 5 decimal places be on the earths surface? Converting the distance in metres between two coordinates provides a way to evaluate the precision of the coordinates on the earths surface. This can be done using basic trigonometry and the shape of the earth as illustrated in the image below. The shape of the earth can be represented as an ellipsoid. The WGS84 ellipsoidal datum, used for GPS positioning, defines the distance of the semi-major axis (a) as 6378137 m. The semi-major axis approximates the distance from the earths centroid to the earths surface at the equator (x). Using the trigonometric equation below, distances can be calculated at the surface of the ellipsoid at various decimal degrees (such as 0.1, 0.01, 0.001 etc) to evaluate how many decimal places should be used to report lat/long.

WGS84_precision The table below shows the distance along the earths surface at the equator that corresponds to the precision of lat/long coordinates, as well as the suitable applications for each precision. For example, reporting lat/long to 0.1 decimal degree, would be equivalent to 360 arc seconds (or 6 arc minutes), and at the equator this spans a distance of 11.1 km. Reporting lat/long in decimal degrees to 4 or 5 decimal places is suitable for most non-GIS or -surveying applications such as web-mapping or recreational GPS. Using 2 or 3 decimal places might be wise if privacy is a concern when sharing coordinates with others. Keeping 6 or more decimal places would be useful for the most precise GIS applications with input from land surveying. The distances change very little across the entire earths surface (test this using the above equation if you are curious).

Precision               (decimal degrees) Precision              (deg-min-sec) Distance at equator (in metres) Application
0.1 0° 6′ 0″ 11,131.96
0.01 0° 0′ 36″ 1,113.195 Global/privacy
0.001 0° 0′ 3.6″ 111.319 To protect privacy
0.0001 0° 0′ 0.36″ 11.131 Handheld GPS
0.00001 0° 0′ 0.036″ 1.113 GIS/Handheld GPS
0.000001 0° 0′ 0.0036″ 0.111 Surveying/GIS
0.0000001 0° 0′ 0.00036″ 0.011 Surveying
0.00000001 0° 0′ 0.000036″ 0.001 Surveying

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s