Support for BIGTIFF/other custom GDAL options

[Andrew Gene Brown]

Thanks for the excellent package! I've been putting it to good use lately.

I noticed that if a download results in a combined GeoTIFF >4GB (after 'deflate' compression), writing to the file with rasterio will result in this GDAL error:

ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.

From reading https://gdal.org/drivers/raster/gtiff.html#creation-options, I see GDAL defaults to BIGTIFF=IF_NEEDED, which incidentally never results in BIGTIFF format file when a compression method is being used.

Since geedim specifies "deflate" compression and other GDAL dataset write options, but not BIGTIFF, lengthy downloads will fail part way through if/when the file reaches ~4GB. I think there probably would be little risk to just setting BIGTIFF='YES' in profile from _prepare_for_download() but perhaps IF_SAFER would be better. It is my understanding that some non-GDAL software still does not support libtiff > 4 / BIGTIFF.

More generally, if the user could decide which (if any) compression method they want to use, or have access to set/modify other GDAL options, then setting BIGTIFF for some data sets/extents would be up to the user

[Dugal Harris]

Hi, thank you for raising this, and for the detective work. I'll address the BIGTIFF issue in the next release. geedim knows the raw size of the raster, so can set BIGTIFF='YES' when it is > 4GB. I think that is probably safest.

I would like to keep the API/CLI as simple as possible. Is there a specific need for user access to the compression type / other GDAL options?

[Andrew Gene Brown]

Great thanks!

Is there a specific need for user access to the compression type / other GDAL options?

No, not at this time for me. And deflate works fine for my purposes.

[Dugal Harris]

Resolved with v1.3.0.