Does Dash Leaflet support FlatGeobuf?
araichev opened this issue · comments
I see that Dash Leaflet supports the Geobuf format.
Does it also support FlatGeobuf, a new performant geospatial file format?
If so, how do i use it?
If not, i'd like to request a feature to support it, please.
Thanks for your attention.
No, only Geobuf and "raw" data (i.e. GeoJSON) are currently supported.
OK, in that case could you file this issue away as feature request?
Cheers.
Just out of curiosity - why do you prefer FlatGeobuf as compared to other formats? I am considering what formats (if any) to support beyond "raw" GeoJSON. There are a lot, but FlatGeobuf seems like one of the more interesting options.
I'd say just pick a format that is very performant and that can be easily processed in python. Geobuf was nice because it covered both of these (I used this for all my input to GeoJSON), but it looks like you've removed geobuf as an input to the GeoJSON component in version 1.
I agree with @mullenkamp and add that FlatGeobuf is such a format.
It also seems to be more performant and better supported than Geobuf, judging by the tables at https://github.com/flatgeobuf/flatgeobuf .
I do hope you re-introduce support in the GeoJSON component for something like FlatGeobuf or Geobuf, because my apps benefit greatly from storing small geodata files.
And following up on this, flatgeobuf seems to be fully supported in Fiona/GDAL. So even though there doesn't seem to be a specific python package for flatgeobuf, python users should be able to directly read/write flatgeobuf to files and file objects using Fiona.
I didn't even know about flatgeobuf until it was mentioned here ;)
Thanks @araichev !
@mullenkamp I originally added geobuf support based on similar requirements. However,
- Through the years, I have experiences non-backwards compatible format changes, i.e. the
.pbffiles generated by some versions of the Python library only worked with some versions of the JSPbflibrary - The Python bindings are no longer actively maintained
Both of these points seem like red flags to me, and combined with requests for other formats (e.g. FlatGeobuf), I figured that now (i.e. the breaking 1.0.0 release) might be the right time to reconsider, which format(s) to support. Ideally, I would like a single (for simplicity and to keep the bundle size minimal) performant format, which is stable, and which has proper binding for both Python and JS.
One downside of FlatGeobuf, as seen from a dash-leaflet perspective is that it doesn't include compression. I understand why they didn't do it (i.e. separation of concerns), but for most use cases in dash-leaflet where the whole data chunk is sent, file size matters. And on this parameter, geobuf is much better (i.e. files are a lot smaller, typically more than 5x). It does however come wither other interesting features, e.g. bounding-box queries via HTTP range requests.
With the two format providing obviously different benefits, I would seem that the "best" solution may be to support both format, including adding range request support for FlatGeobuf. That would take a bit of work though.
Thanks @emilhe for the feedback. I figured you dropped geobuf because the python package doesn't really seem to be actively maintained, but since I am very ignorant of JS development I didn't know about the JS issues (which is totally understandable).
I do see your point related to file size. Is it possible in your example use case when the entire data chunk is sent to leaflet to compress the flatgeobuf file during transfer (using zstd for example)? I'm guessing it would only make a significant difference in larger polygons, but you probably have a better feel for this than me.
Nevertheless, thank again for all your hard work on this. I use dash-leaflet a lot and I'm usually pushing it capabilities pretty hard ;)
With the considerations mentioned in this thread in mind, I have decided to keep geobuf support, and add flatgeobuf support. I have implemented both in the 1.0.8 release. I have also added an example to the docs.
@emilhe ah, i didn't realize FlatGeobuf is missing compression.
Darn.
Yes, because of that i see the logic in implementing both Geobuf and FlatGeobuf support for now.
Thanks!
You're amazing @emilhe ! And you updated the docs too! Thank you very much.
And by the way, the new docs site looks really good ;)
I was just looking at your code for the flatgeobuf implementation, and it looks like flatgeobuf will only work when passed to the GeoJSON component as a url. While geobuf (as it was in the past) can be passed as either url or data. Is this correct?
This is fine by me, I just wanted to make sure I'm understanding the capabilities/limitations.
Thanks again!
Also, I think the smaller size of geobuf is due to the coordinates being assigned a precision and converted to integers rather than any other kind of compression methods applied on the binary data. The default precision is 6 digits.
@mullenkamp Thanks! And yes, it is only supported via url, not via data (I don't see any use case for doing that). I have added a note in the docs to make that more clear.