Esri / coordinate-conversion-addin-dotnet

Addin for ArcMap and Pro for convenient coordinate conversion in Desktop.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Loss of precision when converting from UTM to WGS and then back

cervengoc opened this issue · comments

Hi,

We receive input data in UTM33 format, while storing it in WGS format in the database. Hence we need to convert the UTM33 values to WGS, and then back when displaying them to the user.

We are struggling with some significant loss of precision, which at first glance probably should not happen in my opinion. Maybe we're doing something wrong.

As an example, we have code such as this one.

var northing = 5763616.61290418;
var easting = -603382.328724074;

var coord = UniversalTransverseMercator.ConvertUTMtoLatLong(
    new UniversalTransverseMercator("N", 33, easting, northing));

var lat = coord.Latitude.ToDouble();
var lon = coord.Longitude.ToDouble();

// Convert back

var coord = new CoordinateSharp.Coordinate(point.Y, point.X);
coord.Lock_UTM_MGRS_Zone(33);

var northingAfterConversion = coord.UTM.Northing;
var eastingAfterConversion = coord.UTM.Easting;

// I would expect that at least 1cm precision should be preserved)

northing.Should().BeApproximately(northingAfterConversion, 0.01);
easting.Should().BeApproximately(eastingAfterConversion, 0.01);

The asseritions will fail for us, as the difference is quite significant.

  • northingAfterConversion is 5763616.54513533 (off by ~7 cm)
  • eastingAfterConversion is -603381.519924373 (off by ~80 cm)

Could you please help with this one? What is causing this issue, and how could we increase the back and forth conversion precision?

Sorry for the inconvenience, I posted this to the wrong place by accident. My target was the CoordinateSharp library's repo.