(pdal pipeline readers.copc Error) GDAL failure (1) PROJ: utm: Invalid latitude
kjwaters opened this issue · comments
Describe the bug
I'm running a pipeline with a COPC reader that includes a bounds definition in a different projection than the data. I'm getting an unexpected error reported of invalid latitudes, though I don't think it's actually losing any data.
The error is "(pdal pipeline readers.copc Error) GDAL failure (1) PROJ: utm: Invalid latitude" repeated many times.
$ pdal pipeline -i pipe.json
{
"pipeline":[
{
"type": "readers.copc",
"filename": "https://ocmgeodatastor1.blob.core.windows.net/lidar-copc/529/20070717_098821_w_ld_p31.copc.laz",
"bounds": "([383868.0, 384691], [2950995.0, 2952001])/EPSG:3724"
}
]
}
Expected behavior
The pipeline should not have given any errors or warnings.
If I don't specify the bounds in the reader, but instead add a filters.reprojection for EPSG:3724 and a filters.crop for the same bounds,
I don't see any errors and get the same number of points.
While I don't think this is causing any issues, I'm not positive and it's a bit disconcerting. All the points should be well within the UTM zone.
System/installation information:
Please provide information on your PDAL version (pdal --version
) and system (e.g., uname -a
or ver
).
$ uname -a
Linux 260f4c13d9b3 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
$ pdal --version
pdal 2.7.0 (git-version: d5d146)
If installed via Conda, you may be asked to paste the output of conda list
and conda info
as well.
$ conda list
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
aenum 3.1.15 pypi_0 pypi
archspec 0.2.2 pyhd8ed1ab_0 conda-forge
aws-c-auth 0.7.16 h79b3bcb_6 conda-forge
aws-c-cal 0.6.10 hb29e0c7_1 conda-forge
aws-c-common 0.9.13 hd590300_0 conda-forge
aws-c-compression 0.2.18 hecc5fa9_1 conda-forge
aws-c-event-stream 0.4.2 hf9b2f7b_4 conda-forge
aws-c-http 0.8.1 h5d7533a_5 conda-forge
aws-c-io 0.14.5 h50678d4_1 conda-forge
aws-c-mqtt 0.10.2 hf479d2b_4 conda-forge
aws-c-s3 0.5.2 h4ad9680_0 conda-forge
aws-c-sdkutils 0.1.15 hecc5fa9_1 conda-forge
aws-checksums 0.1.18 hecc5fa9_1 conda-forge
aws-crt-cpp 0.26.2 h19f5d62_7 conda-forge
aws-sdk-cpp 1.11.267 h5606698_1 conda-forge
azure-core-cpp 1.10.3 h91d86a7_1 conda-forge
azure-storage-blobs-cpp 12.10.0 h00ab1b0_0 conda-forge
azure-storage-common-cpp 12.5.0 hb858b4b_2 conda-forge
blosc 1.21.5 h0f2a231_0 conda-forge
boltons 23.1.1 pyhd8ed1ab_0 conda-forge
brotli-python 1.1.0 py310hc6cd4ac_1 conda-forge
bzip2 1.0.8 hd590300_5 conda-forge
c-ares 1.24.0 hd590300_0 conda-forge
ca-certificates 2024.2.2 hbcca054_0 conda-forge
cairo 1.18.0 h3faef2a_0 conda-forge
ceres-solver 2.2.0 h30ec75d_2 conda-forge
certifi 2024.2.2 pyhd8ed1ab_0 conda-forge
cffi 1.16.0 py310h2fee648_0 conda-forge
cfitsio 4.4.0 hbdc6101_0 conda-forge
charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
conda 23.11.0 py310hff52083_1 conda-forge
conda-libmamba-solver 23.12.0 pyhd8ed1ab_0 conda-forge
conda-package-handling 2.2.0 pyh38be061_0 conda-forge
conda-package-streaming 0.9.0 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 pypi_0 pypi
curl 8.5.0 hca28451_0 conda-forge
cycler 0.12.1 pypi_0 pypi
dbf 0.99.9 pypi_0 pypi
debugpy 1.8.1 pypi_0 pypi
distro 1.8.0 pyhd8ed1ab_0 conda-forge
draco 1.5.7 h00ab1b0_0 conda-forge
eigen 3.4.0 h00ab1b0_0 conda-forge
expat 2.6.2 h59595ed_0 conda-forge
fmt 10.1.1 h00ab1b0_1 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_1 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.50.0 pypi_0 pypi
freetype 2.12.1 h267a509_2 conda-forge
freexl 2.0.0 h743c826_0 conda-forge
gdal 3.8.4 py310he073c5f_3 conda-forge
geos 3.12.1 h59595ed_0 conda-forge
geotiff 1.7.1 h6b2125f_15 conda-forge
gflags 2.2.2 he1b5a44_1004 conda-forge
giflib 5.2.1 h0b41bf4_3 conda-forge
glog 0.7.0 hed5481d_0 conda-forge
gmp 6.3.0 h59595ed_1 conda-forge
hdf4 4.2.15 h2a13503_7 conda-forge
hdf5 1.14.3 nompi_h4f84152_100 conda-forge
icu 73.2 h59595ed_0 conda-forge
idna 3.6 pyhd8ed1ab_0 conda-forge
json-c 0.17 h7ab15ed_0 conda-forge
jsonpatch 1.33 pyhd8ed1ab_0 conda-forge
jsonpointer 2.4 py310hff52083_3 conda-forge
kealib 1.5.3 h2f55d51_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.5 pypi_0 pypi
krb5 1.21.2 h659d440_0 conda-forge
laspy 2.5.3 pypi_0 pypi
lcms2 2.16 hb7c19ff_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libabseil 20230802.1 cxx17_h59595ed_0 conda-forge
libaec 1.1.3 h59595ed_0 conda-forge
libarchive 3.7.2 h2aa1ff5_1 conda-forge
libblas 3.9.0 21_linux64_openblas conda-forge
libboost-headers 1.84.0 ha770c72_1 conda-forge
libcblas 3.9.0 21_linux64_openblas conda-forge
libcrc32c 1.1.2 h9c3ff4c_0 conda-forge
libcurl 8.5.0 hca28451_0 conda-forge
libdeflate 1.19 hd590300_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 hd590300_2 conda-forge
libexpat 2.6.2 h59595ed_0 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 13.2.0 h807b86a_3 conda-forge
libgdal 3.8.4 hab4ef92_3 conda-forge
libgfortran-ng 13.2.0 h69a702a_5 conda-forge
libgfortran5 13.2.0 ha4646dd_5 conda-forge
libglib 2.80.0 hf2295e7_1 conda-forge
libgomp 13.2.0 h807b86a_3 conda-forge
libgoogle-cloud 2.17.0 h884d01b_1 conda-forge
libgoogle-cloud-storage 2.17.0 hc7a4891_1 conda-forge
libgrpc 1.59.3 hd6c4280_0 conda-forge
libhwloc 2.9.3 default_h554bfaf_1009 conda-forge
libiconv 1.17 hd590300_2 conda-forge
libjpeg-turbo 3.0.0 hd590300_1 conda-forge
libkml 1.3.0 h01aab08_1018 conda-forge
liblapack 3.9.0 21_linux64_openblas conda-forge
libmamba 1.5.5 had39da4_0 conda-forge
libmambapy 1.5.5 py310h39ff949_0 conda-forge
libnetcdf 4.9.2 nompi_h9612171_113 conda-forge
libnghttp2 1.58.0 h47da74e_1 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libopenblas 0.3.26 pthreads_h413a1c8_0 conda-forge
libpng 1.6.43 h2797004_0 conda-forge
libpq 16.2 h33b98f1_1 conda-forge
libprotobuf 4.24.4 hf27288f_0 conda-forge
libre2-11 2023.09.01 h7a70373_1 conda-forge
librttopo 1.1.0 h8917695_15 conda-forge
libsolv 0.7.27 hfc55251_0 conda-forge
libspatialite 5.1.0 h7bd4643_4 conda-forge
libsqlite 3.45.2 h2797004_0 conda-forge
libssh2 1.11.0 h0841786_0 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_3 conda-forge
libtiff 4.6.0 ha9c0a0a_2 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libwebp-base 1.3.2 hd590300_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxml2 2.12.6 h232c23b_0 conda-forge
libzip 1.10.1 h2629f0a_3 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
lzo 2.10 h516909a_1000 conda-forge
mamba 1.5.5 py310h51d5547_0 conda-forge
matplotlib 3.8.2 pypi_0 pypi
menuinst 2.0.1 py310hff52083_0 conda-forge
metis 5.1.0 h59595ed_1007 conda-forge
minizip 4.0.5 h0ab5242_0 conda-forge
mpfr 4.2.1 h9458935_0 conda-forge
ncurses 6.4 h59595ed_2 conda-forge
nitro 2.7.dev8 h59595ed_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.98 h1d7d5a4_0 conda-forge
numpy 1.26.4 py310hb13e2d6_0 conda-forge
openjpeg 2.5.2 h488ebb8_0 conda-forge
openssl 3.2.1 hd590300_1 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pcre2 10.43 hcad00b1_0 conda-forge
pdal 2.7.0 h312035a_0 conda-forge
pdal-plugins 1.3.0 pypi_0 pypi
pillow 10.2.0 pypi_0 pypi
pip 23.3.2 pyhd8ed1ab_0 conda-forge
pixman 0.43.2 h59595ed_0 conda-forge
platformdirs 4.1.0 pyhd8ed1ab_0 conda-forge
pluggy 1.3.0 pyhd8ed1ab_0 conda-forge
poppler 24.03.0 h590f24d_0 conda-forge
poppler-data 0.4.12 hd8ed1ab_0 conda-forge
postgresql 16.2 h82ecc9d_1 conda-forge
proj 9.3.1 h1d62c97_0 conda-forge
proj-data 1.17 hd8ed1ab_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pybind11-abi 4 hd8ed1ab_3 conda-forge
pycosat 0.6.6 py310h2372a71_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pyodbc 5.1.0 pypi_0 pypi
pyparsing 3.1.2 pypi_0 pypi
pyproj 3.6.0 pypi_0 pypi
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.10.13 hd12c33a_0_cpython conda-forge
python-dateutil 2.9.0.post0 pypi_0 pypi
python-dotenv 1.0.0 pypi_0 pypi
python-pdal 3.3.0 py310he6b3783_6 conda-forge
python_abi 3.10 4_cp310 conda-forge
rclone 1.65.2 h519d9b9_0 conda-forge
re2 2023.09.01 h7f4b329_1 conda-forge
readline 8.2 h8228510_1 conda-forge
reproc 14.2.4.post0 hd590300_1 conda-forge
reproc-cpp 14.2.4.post0 h59595ed_1 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
ruamel.yaml 0.18.5 py310h2372a71_0 conda-forge
ruamel.yaml.clib 0.2.7 py310h2372a71_2 conda-forge
s2n 1.4.5 h06160fa_0 conda-forge
saxonche 12.4.2 pypi_0 pypi
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
shapely 2.0.1 pypi_0 pypi
six 1.16.0 pypi_0 pypi
snappy 1.1.10 h9fff704_0 conda-forge
sqlite 3.45.2 h2c6b66d_0 conda-forge
suitesparse 5.10.1 h5a4f163_3 conda-forge
tbb 2021.11.0 h00ab1b0_1 conda-forge
tiledb 2.20.1 hd75ad12_1 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
tqdm 4.66.1 pyhd8ed1ab_0 conda-forge
truststore 0.8.0 pyhd8ed1ab_0 conda-forge
tzcode 2024a h3f72095_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
uriparser 0.9.7 h59595ed_1 conda-forge
urllib3 2.1.0 pyhd8ed1ab_0 conda-forge
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
xerces-c 3.2.5 hac6953d_0 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.7 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml-cpp 0.8.0 h59595ed_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zstandard 0.22.0 py310h1275a96_0 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge
$ conda info
I don't have an answer, just posting some debugging info and ideas.
When I run the pipeline with PROJ_DEBUG=3
set, I get the following:
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: pj_ellipsoid - final: ellps=GRS80
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: pj_ellipsoid - final:
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: Building arg list for step no. 0
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: init - proj=axisswap, 2
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: order=2,1
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: axisswap: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: axisswap: pj_ellipsoid - final: ellps=GRS80
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: Step 0 (proj=axisswap) at 0x1122b5c80
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline at [0x1122b6080]: step at [0x1122b5c80] (proj=axisswap) done
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: Building arg list for step no. 1
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: init - proj=unitconvert, 3
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: xy_in=deg
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: xy_out=rad
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: unitconvert: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: unitconvert: pj_ellipsoid - final: ellps=GRS80
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: unitconvert: xy_in unit: Degree
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: unitconvert: xy_out unit: Radian
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: Step 1 (proj=unitconvert) at 0x1122b5880
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline at [0x1122b6080]: step at [0x1122b5880] (proj=unitconvert) done
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: Building arg list for step no. 2
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: init - proj=utm, 3
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: zone=17
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: ellps=GRS80
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: utm: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: utm: pj_ellipsoid - final: ellps=GRS80
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: Step 2 (proj=utm) at 0x1122b5480
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline at [0x1122b6080]: step at [0x1122b5480] (proj=utm) done
(pdal pipeline readers.copc Debug) GDAL debug: PROJ_TRACE: pipeline: Pipeline: 3 steps built. Determining i/o characteristics
COPC of curvilinear coordinate systems, like this file has, is not such a great idea. Your octree ends up looking awful. See @connormanning's moonlidar.com talk for some consequences.
(pdal pipeline readers.copc Debug) 18 overlapping nodes
IMO it would be better to write the data into a rectilinear 3D coordinate system with the units all being the same. Some kind of ECEF like was done with moonlidar.com is one way to do it.
ECEF coordinate systems aren't so friendly for people for other reasons though. There's no silver bullet.