Upgrade to PROJ 8.2.0
kbevers opened this issue · comments
There's a new PROJ release out, WEBPROJ should use it once it has had its first maintenance release. Adopting the new version should be simple but there may be a few issues regarding grids that may need to be handled differently.
When building the docker image grids can now be pulled from https://cdn.proj.org/ which should speed things up a bit.
8.2.1 is out, so we should perhaps look into upgrading :)
Give it a go
PROJ 8.2.0 is built into pyproj 3.3.0, which in return requires Python 3.8.
So, I'll start by trying to get WEBPROJ to run on Python 3.8+
#31 updates to Python 3.8. Tested locally.
#31 updates to Python 3.8. Tested locally.
Cool. Any issues regarding the PROJ update?
I have run into some issues with trying to install pyproj 3.3.0 with conda, but it downloads a 2.x version. I'll figure it out :)
pyproj-3.3.0 | 425 KB | ########## | 100%
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
(base)
B0XXXXX@CXXXXXXX MINGW64 ~/PycharmProjects/WEBPROJ (master)
$ pytest
============================= test session starts =============================
platform win32 -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\b025966\PycharmProjects\WEBPROJ
collected 25 items
tests\test_api.py ......................... [100%]
============================= 25 passed in 8.08s ==============================
(base)
B0XXXXX@CXXXXXXX MINGW64 ~/PycharmProjects/WEBPROJ (master)
$
So it seems like it works, even without installing the transformation grids(?)
PROJ will by default try to download grids from https://cdn.proj.org if they aren't available on the local system. Great for casual use, not ideal in this scenario.
projsync is the canonical way to download grids these days but a simple curl or wget call should do the trick as well.
The grid downloading from OSGeo is terribly slow - 10 minutes and counting. Might I suggest we store them in the repo for faster builds?
The grid downloading from OSGeo is terribly slow - 10 minutes and counting.
Can't reproduce:
$ time projsync --target-dir projtmp --source-id dk_sdfe
Downloading from https://cdn.proj.org into projtmp
Total size to download: 9 MB
Downloading https://cdn.proj.org/dk_sdfe_README.txt... (1 / 14)
Downloading https://cdn.proj.org/DK... (2 / 14)
Downloading https://cdn.proj.org/FO... (3 / 14)
Downloading https://cdn.proj.org/dk_sdfe_DK_bornholm.pol... (4 / 14)
Downloading https://cdn.proj.org/dk_sdfe_DK_bridges.pol... (5 / 14)
Downloading https://cdn.proj.org/dk_sdfe_DK_general.pol... (6 / 14)
Downloading https://cdn.proj.org/dk_sdfe_DK_jutland.pol... (7 / 14)
Downloading https://cdn.proj.org/dk_sdfe_DK_zealand.pol... (8 / 14)
Downloading https://cdn.proj.org/dk_sdfe_FO_fk89.pol... (9 / 14)
Downloading https://cdn.proj.org/dk_sdfe_dnn.tif... (10 / 14)
Downloading https://cdn.proj.org/dk_sdfe_dvr90.tif... (11 / 14)
Downloading https://cdn.proj.org/dk_sdfe_fvr09.tif... (12 / 14)
Downloading https://cdn.proj.org/dk_sdfe_gvr2000.tif... (13 / 14)
Downloading https://cdn.proj.org/dk_sdfe_gvr2016.tif... (14 / 14)
projsync --target-dir projtmp --source-id dk_sdfe 0.29s user 0.16s system 12% cpu 3.509 total
four or five seconds tops. Don't download everything, just the grids we need. As far as I remember we only use our own at the moment.
If this is a genuine concern then cache the files before building the docker image. Should be easy with GitHub Actions.
The grid downloading from OSGeo is terribly slow - 10 minutes and counting.
Can't reproduce:
That's from cdn.proj.org - not the https://download.osgeo.org/proj/ URL we use in the Docker image.
Anyways, pull request incoming soon. It does seem to work.
That's from cdn.proj.org - not the https://download.osgeo.org/proj/ URL we use in the Docker image.
Hence this comment :-)
projsync is the canonical way to download grids these days but a simple curl or wget call should do the trick as well.
I made the Dockerfiles before the PROJ CDN was set up. The CDN should be more reliable and faster.
OK, I'll open a new issue for switching to Proj's CDN then :)