gdal_calculations does not handle rasters with positive N-S pixel resolutions (was Invalid dataset dimenions: 498 x -435)
GoogleCodeExporter opened this issue · comments
I'm having some trouble getting gdal_calculations to work reliably with some
inputs that I think should be working properly. It works great on occasion, but
usually I get an error like:
RuntimeError: Invalid dataset dimensions : 498 x -435
Using those same two inputs in the raster calculator in QGIS with the same
formula, it produces the expected output.
Command I'm running:
/usr/bin/python -m gdal_calculations --A /tmp/rastercalc7LCtaE.tif --B
/tmp/rastercalc9lrqbS.tif --extent MAXOF --reproject --outfile
/tmp/rastercalcZRb4Dh.tif --calc='(A[0] * B[0])'
I've attached the two files. They overlap nicely in QGIS, where the calculation
works as expected. If it matters, I'm using GDAL 1.11.0 with Python 2.7.3.
Can you provide any guidance? Am I using it incorrectly, or is this a bug of
some kind?
I'd like to use this as part of an automated process (hence the weird
filenames), so having it highly reliable would be important to me. If it helps,
I could provide a limited amount of funding for the support/consulting on this
issue. Probably not a ton, but enough for a nice dinner or two at least!
You can reach me directly at chris (at) strategiccode.com if needed.
Thank you!
Chris Michaelis
PS. More output, if useful:
Python 2.7.3 (default, Feb 27 2014, 20:00:17)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import osgeo.gdal
>>> print osgeo.gdal.__version__
1.11.0
>>>
/usr/bin/python -m gdal_calculations --A /tmp/rastercalc7LCtaE.tif --B
/tmp/rastercalc9lrqbS.tif --extent MAXOF --reproject --outfile
/tmp/rastercalcZRb4Dh.tif --calc='(A[0] * B[0])' 2>&1
Running calculation
0Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/__main__.py", line 25, in <module>
gdal_calculations.gdal_calculate.main()
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/gdal_calculate.py", line 235, in main
outfile = eval(args.calc)
File "<string>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/gdal_dataset.py", line 397, in __mul__
return self.__operation__(operator.__mul__,other)
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/gdal_dataset.py", line 338, in __operation__
dataset1,dataset2=self.check_extent(other)
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/gdal_dataset.py", line 80, in apply_environment
dataset1,dataset2=self.__check_extent__(dataset1,dataset2)
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/gdal_dataset.py", line 184, in __check_extent__
if dataset1.extent!=ext: dataset1=ClippedDataset(dataset1,ext)
File "/usr/local/lib/python2.7/dist-packages/gdal_calculations/gdal_dataset.py", line 713, in __init__
self._dataset=gdal.Open(self._filename)
RuntimeError: Invalid dataset dimensions : 498 x -435
Original issue reported on code.google.com by cmichae...@gmail.com on 8 Jan 2015 at 7:45
Attachments:
Your images have a positive N-S pixel resolution in the geotransform. This is
quite unusual and means the origin coordinates for the image are in the
lower-left rather than the upper-left of the image. In the usual upper-left
case the geotransform N-S pixel resolution would be negative.
http://www.gdal.org/gdal_tutorial.html#gdal_tutorial_dataset
Are you sure your image georeferences are correct?
Original comment by lukepinn...@gmail.com on 9 Jan 2015 at 2:59
Hi Luke,
Thanks for taking a look at this. That's very odd - the data was all generated
by GDAL utilities, nothing of my own creation. The process leading up to those
was basically...
1) Take input CSV file -
x,y,z
...values in lat/long...
2) Create a VRT file to make it readable as a data source, using
PointFromColumns encoding for the GeometryField
<OGRVRTDataSource>
<OGRVRTLayer name="in">
<SrcDataSource>in.csv</SrcDataSource>
<GeometryType>wkbPoint</GeometryType>
<GeometryField encoding="PointFromColumns" x="x" y="y" z="z"/>
</OGRVRTLayer>
</OGRVRTDataSource>
3) Reproject from 4326 to 900913:
/usr/bin/ogr2ogr -nlt POINT25D -t_srs EPSG:900913 -s_srs EPSG:4326 -f CSV -lco
GEOMETRY=AS_XYZ out.csv in.vrt
4) Repeat step 2 for new file...
5) Use that VRT to interpolate to a GeoTIFF:
/usr/bin/gdal_grid -quiet -a invdist:power=2.0:smoothing=0 -of GTiff -ot
Float64 -l out out.vrt out.tif
6) Then send it over to gdal_calculate!
Using gdalinfo against the files led me to believe everything was solid, backed
up by having it render as it should in QGIS of course (I tend to use QGIS to
sanity check outputs). I can't imagine the GDAL tools themselves messing up the
geotransform that severely, but maybe something is going awry in that setup
process...I'll experiment a bit more with the "prep work" once I'm back in the
office.
You said "quite unusual" -- does that translate to "invalid"? Just curious. :)
Thanks again!
Chris
Original comment by cmichae...@gmail.com on 9 Jan 2015 at 3:15
[deleted comment]
Original comment by lukepinn...@gmail.com on 9 Jan 2015 at 4:06
- Changed title: gdal_calculations does not handle rasters with positive N-S pixel resolutions (was Invalid dataset dimenions: 498 x -435)
No, it's not invalid so far as I'm aware. It's just very unusual, I've never
come across a bottom up referenced image in over 15 years of GIS work.
Apparently it's not so uncommon with certain formats like NetCDF.
The code should handle it, it doesn't so it is a bug.
I have an idea for a temporary workaround (based on this ticket -
http://trac.osgeo.org/gdal/ticket/3986). Can you provide a sample of the XYZ
csv?
Original comment by lukepinn...@gmail.com on 9 Jan 2015 at 4:08
Back in my office! I dug a little closer and see now what you meant - yeah,
it's "upside down", and then has a negative pixel size, which is technically
valid, but...weird.
I read that ticket you linked to, and that's definitely what's going on. I
guess it's just a quirk of the XYZ driver for reading CSV (VRT) data as a grid
data source. However, that ticket contained an invaluable hint -- just piping
it through gdalwarp will "correct" the file so that it's what sane people would
expect.
I tested just popping in a gdalwarp step into my process here, and works like
magic. All better!
So, using that workaround, I'm up and running just fine. If you'd still like to
make gdal_calculate handle negative pixel height, it'd probably make it more
resilient - but I'm good as is, with your hint and point in the right
direction. Just in case you do still want the XYZ csv data example, I've
attached it here. The CSV and VRT were just written by fwrite() and are nothing
special -- then after those were written, the process was:
gdal_grid -quiet -a invdist:power=2.0:smoothing=0 -of GTiff -ot Float64 -l
rastercalcyL3a3r /tmp/rastercalcyL3a3r.vrt /tmp/rastercalc298oaM.tif
gdal_grid -quiet -a invdist:power=2.0:smoothing=0 -of GTiff -ot Float64 -l
rastercalchKUIrr /tmp/rastercalchKUIrr.vrt /tmp/rastercalc5qrP3L.tif
# Le workaround...
gdalwarp /tmp/rastercalc298oaM.tif /tmp/A.tif
gdalwarp /tmp/rastercalc5qrP3L.tif /tmp/B.tif
python -m gdal_calculations --A /tmp/A.tif --B /tmp/B.tif --extent MAXOF
--reproject --outfile /tmp/out.tif --calc='(A[0] * B[0])'
Cheers,
Chris
Original comment by cmichae...@gmail.com on 9 Jan 2015 at 6:56
Attachments:
- [Test Files.zip](https://storage.googleapis.com/google-code-attachments/gdal-calculations/issue-8/comment-6/Test Files.zip)
Yep, gdalwarp was what I was going to try. Thanks for the sample.
Original comment by lukepinn...@gmail.com on 9 Jan 2015 at 7:59
This issue was closed by revision r171.
Original comment by lukepinn...@gmail.com on 12 Feb 2015 at 6:52
- Changed state: Fixed
New version released with this fix - http://goo.gl/MsMXj4
Original comment by lukepinn...@gmail.com on 12 Feb 2015 at 7:42