Deprecate use of negative IDs
joto opened this issue · comments
Looking at the code I can see that osm2pgsql has provisions for the use of negative node/way/relation IDs in some places. But I am unsure whether this works correctly. Maybe it does in some circumstances and not in others. One problem is the use of negative IDs in the database for areas created from multipolygon relations. This will clash with negative way IDs.
Catering for negative IDs makes the code more complex and at least some of the code looks like it uses extra memory to store those IDs that almost no user will have. Do we actually need this and want to support it? What are the use cases? Are there uses that can't be replaced by using osmium renumber to first create a file without negative IDs?
Negative IDs are commonly used when handling external data that doesn't come from OSM.org but needs to be stored alongside public data with positive IDs for rendering or conflation purposes. It’s helpful to have an integer range that’s known to be free of legitimate OSM.org entities.
But I am unsure whether this works correctly
I don't think they do.
Negative IDs are commonly used when handling external data that doesn't come from OSM.org but needs to be stored alongside public data with positive IDs for rendering or conflation purposes. It’s helpful to have an integer range that’s known to be free of legitimate OSM.org entities.
Are you doing this successfully with osm2pgsql? Have you evaluated if it works when the absolute value of an id conflicts? Does it work with updates?
I’ve not done this with osm2pgsql, but I have used negative ranges in pipelines with other toolchains. I’ve seen the pattern at just about every OSM-using engineering org I’ve been a part of, making me think it’s either common or obvious enough to support.
I've seen people using some ridiculously large positive offsets to handle custom data, which sometimes triggers unexpected issues for some tools (e.g. see this post). Working with carefully chosen positive offsets should work in general, but requires careful end-to-end testing.
@joto This issue about deprecating support of negative IDs in the input files, not in the "osm_id" column of the tables?
@Nakaner Yes, it's about the input files.
If it is about input files only, I do not have any objections against the deprecation.
https://github.com/openstreetmap/osm2pgsql/blob/master/src/node-ram-cache.hpp#L90-L94 still mentions "negative ids". This might be a left over from when negative ids were still allowed. It doesn't make a huge difference, though, if you're dealing with really large object ids.