nibosco / alti3d-downloader

A tool for downloading and processing of DEM data from swissALTI3D

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Downloading and processing tool for working with swissALTI3D data πŸ“ πŸ‘‰ πŸ“

This Python script downloads GeoTIFF files from a list of URLs generated by swissALTI3D, merges them and optionally processes them.

Background πŸ“–

The Swiss Federal Office of Topography swisstopo provides a great source for a digital elevation model (without vegetation and buildings). In my use case, working in an planning office in the field of infrastructure management, we use this data as part of our data validation process during the work on "Genereller EntwΓ€sserungsplan (GEP)" (en: General drainage planning GDP) for various municipalities in Switzerland. A practicable process for data acquisition and pre-processing to integrate the data in our tech stack was created with this tool.

Usage

Deployment 🚒

The script can be deployed inside a docker container. The needed files are provided. Before running docker compose configure the tool via the .env file (see here). There you can specify the folder path which you want to bind to the working directory.

docker compose up -d

Export csv from swisstopo πŸ“€

First export a list of the relevant area on the website of swissALTI3D. Use Cloud Optimized GeoTIFF as the format. Download the generated csv-file containing all the links for the specified tiles. Follow their terms of use.

Workflow πŸ“

Copy the exported csv-file in to the input folder.
The script constantly monitors the input folder inside the working directory. When a csv-file (export from swissALTI3D) is moved into the input folder, the script picks it up, creates a project folder named after the .csv-file (e.g. i use city_name.csv) downloads each GeoTIFF-files into the raw folder. After completion all single GeoTIFF files are merged into one (e.g. city_name_merged.tif) in the project directory. If the optional processing (CONVERT_TO_XYZ) is set to True, the corresponding steps are executed.

Result πŸ“‹

After completion you have a project folder containing the raw data as well as a merged GeoTIFF and optionally a conversion of that into an ASCII XYZ file.

Here is a representation of the folder srtructure.

πŸ“¦alti3d-downloader
 ┣ πŸ“‚input
 ┃ β”— πŸ“˜  city_name.csv
 ┣ πŸ“‚city_name
 ┃ ┣ πŸ“‚raw
 ┃ ┃ ┣ πŸ“— swissalti3d_[..].tif
 ┃ ┃ β”— ... 
 ┃ ┣ πŸ“— city_name_merged.tif
 ┃ β”— πŸ“— city_name_merged.xyz
 ┣ πŸ“„ .dockerignore
 ┣ πŸ“„ .env
 ┣ πŸ“„ docker-compose.yml
 ┣ πŸ“„ dockerfile
 ┣ πŸ“„ download_manager.py
 ┣ πŸ“„ readme.md
 β”— πŸ“„ requirements.txt

 Legend: πŸ“˜ input
         πŸ“— output
 

Configuration βš™οΈ

.env πŸ“„

All config parameters can be changed in the .env file.
Define the working directory for the application using the DIR variable.
CONVERT_TO_XYZ enables/disables optional converting the final GeoTIFF to an ASCII XYZ file.

#set working directory
DIR="../alti3d-downloader"
#optional conversion of the merged GeoTIFF
CONVERT_TO_XYZ=False #default is True

Outlook πŸ”­

Necessity adding downsampling option (swisstopo offers resolution in 0.5 m and 2 m)
Implementation generating a TIN from raster data

Appendix πŸ“š

XYZ Conversion

In our current techstack we need both file formats (.tif as well as .xyz).
While swisstopo provides the option on their site to download .xyz files as well, to avoid double downloading raw data, a conversion is used. The conversion is done by using gdal_translate. The following parameters are used:

gdal_translate -of XYZ -a_srs EPSG:2056 -co "DECIMAL_PRECISION=2"

About

A tool for downloading and processing of DEM data from swissALTI3D


Languages

Language:Python 100.0%