Document vector_layers extension

Question

Document vector_layers extension

pnorman opened this issue 10 years ago · comments

We're starting to see a lot of tilejson with a vector_layers key. We should document this for standardization and so that the meaning the keys are documented.

Some examples

https://gist.github.com/springmeyer/bbf49c89180485f477b9
https://github.com/mapbox/mapbox-studio/blob/mb-pages/test/fixtures-mapboxapi/mapbox.mapbox-streets-v5.json

Edit: fixed link

Anand Thakker · Answer 1 · Fri Oct 09 2015 22:54:14 GMT+0800 (China Standard Time)

Bumping this. Happy to put in a PR if yall agree/confirm that this should happen.

Assuming yes, my understanding is that the schema for vector_layers would look something like:

"vector_layers": {
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
       "id": { "type": "string" }
       "description": { "type": "string" },
       "fields": { "type": "object" }
       "required": [ "id", "fields" ]
    }
  }
}

And the fields object maps field names to descriptions.

Amanda · Answer 2 · Mon Mar 21 2016 19:43:18 GMT+0800 (China Standard Time)

I recently got stung by this when trying to use tessera which uses tilelive to serve up a tmstyle. If you have no vector_layers key in the TileJSON, then the resultant image is blank.

It would be helpful if the spec included this.

Paul Norman · Answer 3 · Sat Jul 16 2016 00:35:34 GMT+0800 (China Standard Time)

ping on this. I could also do a PR if there's agreement and understanding on what the schema is (though if @anandthakker already has something written, I'd prefer that)

Anand Thakker · Answer 4 · Sat Jul 16 2016 01:37:24 GMT+0800 (China Standard Time)

@pnorman Nothing written, alas. Still willing to do it, but my queue is somewhat longer today than it was when I last commented here.

Gregor MacLennan · Answer 5 · Tue Sep 20 2016 07:43:13 GMT+0800 (China Standard Time)

It seems like the only property that mapbox-gl-js currently uses is the id. https://github.com/mapbox/mapbox-gl-js/blob/93c3b030726ff4d22dd9fbd24ce19c27c3f19093/js/style/style.js#L80-L112

Reviewing the source it looks like if that field is not present, and does not map to the layerId in the vector tile, then nothing will render.

Alexander Rolek · Answer 6 · Sat Oct 01 2016 07:41:24 GMT+0800 (China Standard Time)

I'm currently implementing the TileJSON spec for an MVT server and would also like to see an agreed upon spec for the vector_layers key. I came up with the following proposal based on what is required by the MVT Layer Specification and some additional fields I see beneficial for displaying layers. A few things to note:

id and name are redundant. The MVT spec requires a layer have a name value, but as @gmaclennan pointed out, the id value is already being used by mapbox-gl. Ideally the spec would only use name as that's required for MVT Layers.
The fields key as proposed has been renamed to feature_tags. The MVT spec uses tags for feature metadata, and since the values are for "features" not the layer, I think the "feature_" prefix is more explicit.

Here's the spec I'm proposing:

{
    // REQUIRED. The MVT encoding version.
    "version": 2,

    // REQUIRED. The extent of the vector layer. 
    "extent": 4096,

    // REQUIRED. The name of the layer 
    // "name" and "id" are identical since mapbox has already implemented the id in mapbox-gl
    "id": "buildings",

    // REQUIRED. The name of the layer.
    // "name" and "id" are identical since mapbox has already implemented the id in mapbox-gl
    "name": "buildings",

    // OPTIONAL. Default: []
    // an array of feature tags that MAY be included on each feature
    "feature_tags": ["class", "name_en", "etc"]

    // OPTIONAL. Default: null
    // possible values include: "point", "line", "polygon", "unknown"
    "geometry_type": "polygon"

    // OPTIONAL. Default: 0. >= 0, <= 22.
    // An integer specifying the minimum zoom level.
    "minzoom": 0,

    //  OPTIONAL. Default: 22. >= 0, <= 22.
    // An integer specifying the maximum zoom level. MUST be >= minzoom.
    "maxzoom": 11
}

Looking forward to the discussion.

Paul Norman · Answer 7 · Wed Oct 05 2016 21:13:29 GMT+0800 (China Standard Time)

version and extent are specific to MVT, not vector tiles. If we were to specify format it should go a level higher in the json
I assume the json object you are describing is put in an array for the vector_layers property? This needs to be explicit
👎 to having a duplicate name and id properties
null must be a possible geometry_type or leaving it undefined violates the possible values
what's the difference between null geometry_type and unknown?
what about other geometry types?

Alexander Rolek · Answer 8 · Thu Oct 06 2016 03:37:31 GMT+0800 (China Standard Time)

version and extent are specific to MVT, not vector tiles. If we were to specify format it should go a level higher in the json

This brings up an interesting point. The spec needs to be polymorphic to support the various vector formats. I have drafted a new proposal at the end of the comment with this in mind.

I assume the json object you are describing is put in an array for the vector_layers property? This needs to be explicit

Correct. I was assuming context as the issue is about the vector_layers property, but it should be explicit.

👎 to having a duplicate name and id properties

I agree, which is why I suggested name but as @gmaclennan pointed out mapbox is already using the id field. We could always use id and then for MVT implementations put the name into the id field. It will probably be important to document how various formats should leverage this spec.

null must be a possible geometry_type or leaving it undefined violates the possible values

what's the difference between null geometry_type and unknown?

I was trying to match the rest of the TileJSON spec which uses default values of null or [ ]. The geometry values I included were from section 4.3.4 - Geometry Types of the MVT Specification. The UNKNOWN type is used for experimental types.

what about other geometry types?

I was referencing the MVT spec, but let's expand the available values. WKB has 18 geometry types, but I'm not sure what geometry types are supported across all vector formats. Any input on this?

Here's an additional proposal we can pick apart:

"vector_layers":[
    {
        // REQUIRED. The "id" or "name" of the layer 
        "id": "buildings",

        // REQUIRED. The vector tile type.
        "type": "mvt",

        // OPTIONAL. Default: null
        The details specific to the vector layer "type". The following example is for MVT.
        "details": {

            // REQUIRED. The MVT encoding version.
            "version": 2,

            // REQUIRED. The extent of the vector layer. 
            "extent": 4096,

            // OPTIONAL. Default: []
            // an array of feature tags that MAY be included on each feature
            "feature_tags": ["class", "name_en", "etc"]
        },

        // OPTIONAL. Default: null
        // possible values include: "point", "line", "polygon", "unknown", ... [additional values under review].
        "geometry_type": "polygon"

        // OPTIONAL. Default: 0. >= 0, <= 22.
        // An integer specifying the minimum zoom level.
        "minzoom": 0,

        //  OPTIONAL. Default: 22. >= 0, <= 22.
        // An integer specifying the maximum zoom level. MUST be >= minzoom.
        "maxzoom": 11
    }
]

Gregor MacLennan · Answer 9 · Fri Oct 07 2016 05:36:17 GMT+0800 (China Standard Time)

Some further info from experimenting: Mapbox Studio expects the fields property to be present, as an object with field names as keys and field descriptions as values. If this is not present the Tileset view of the webapp crashes. The style editor view works, but it is not possible to filter the layer.

Alexander Rolek · Answer 10 · Sat Oct 08 2016 03:08:58 GMT+0800 (China Standard Time)

@gmaclennan can you provide an example? I wonder how set in stone the Mapbox implementation is? They might have just made some decisions expecting for the spec to drive some changes in the future.

Paul Norman · Answer 11 · Mon Oct 10 2016 06:39:20 GMT+0800 (China Standard Time)

I'm still 👎 on including the format as a property of the vector_layers object.

We should use name to be consistent with the rest of the spec, not id.

I'd like to see vector_layers contain a list of layers, each layer with a name, optional list of fields and min/max zoom where that layer MAY be found. I want to avoid requiring layers to follow a GeoJSON model of Point, LineString, Polygon or GeometryCollection, as it's not the only one that will get used.

Stefan · Answer 12 · Sun Jan 28 2018 22:06:56 GMT+0800 (China Standard Time)

Sorry for insisting but e.g. #29 is closed just because it's referring to this overdue open issue.

Allow me to make an understatement: The lack of support from Mapbox of the TileJSON and the mbtiles specs. for vector tiles contrasts IMHO little bit to its PR activities and $164 million funding.

Can someone from Mapbox tell us who is the maintainer of this mentioned specs.?

IMHO many metadata discussions could be much more efficient if specs. would be updated. See currently openmaptiles/openmaptiles#379 (comment) , or geometalab/Vector-Tiles-Reader-QGIS-Plugin#170 , or systemed/tilemaker#102 .

Recently hope was raising since there's a pull request for mbtiles v1.3 mapbox/mbtiles-spec#46 (comment) .

This hope was put in question when @flippmoke wrote:

After some discussion with @ericfischer and @springmeyer on this issue
last week we have decided to hold off on this until after #47 is complete.

and @pnorman answered

I mentally gave up on the mbtiles and tilejson specs a few months ago

This frustration is the same reason for me hesitating to contribute. But I just don't want to give up yet!

Erica Fischer · Answer 13 · Tue Jan 30 2018 08:52:09 GMT+0800 (China Standard Time)

Sorry for the long delay on spec revisions @sfkeller. I think mapbox/mbtiles-spec#47 is ready to merge to describe the current reality. Please let me know if you have any significant objections to what is there.

Stefan · Answer 14 · Wed Jan 31 2018 23:50:32 GMT+0800 (China Standard Time)

@ericfischer : Thanks. We're reviewing the spec revision now. It's really not just me waiting for this update (also of TileJSON) it's for the sake of innovation based on established standards, see e.g. this from @tomchadwin https://twitter.com/tomchadwin/status/958646784439607296 .

Stefan · Answer 15 · Tue Feb 06 2018 17:57:10 GMT+0800 (China Standard Time)

I'm glad to see mbtiles 1.3 released! https://github.com/mapbox/mbtiles-spec
Now I hope there will be soon a similar update of this TileJSON spec.!

Sam Matthews · Answer 16 · Wed Apr 04 2018 04:34:48 GMT+0800 (China Standard Time)

Hey y'all, thanks for the back-and-forth here. Our plan is to include vector_layers in the next version of the specification (version 3). Since this key is clearly being used in Mapbox and other implementations, it needs to be documented. @GretaCB and myself will comment back here as we start drafting the next version.

Sam Matthews · Answer 17 · Sat May 05 2018 05:03:00 GMT+0800 (China Standard Time)

Per discussion with @ericfischer - the current plan in documenting the structure of vector_layers is to take the text from the mbtiles spec and move it into this specification as a source of truth, therefore the mbtiles-specification can point directly to version 3.0 here, rather than documenting the same thing in two places.