Extract nested structs

[andoks]

Currently all nested structs are generated as literal structs within the parent struct identical to the data being parsed. Is it possible to make go-jsonstruct extract such nested struct definitions to named structs which type is then used as a field instead of the nesting of the struct literals? This would make it easier in some cases to create instances for posting to REST-api's, as otherwise I have to repeat the nested type definition when specifying the value.

E.g based on input

{
  "items": [
    {
      "datapoints": [
        {
          "timestamp": 1687907013216,
          "value": 42.54
        },
        {
          "timestamp": -149967180845,
          "value": 1337.43
        }
      ],
      "id": 6361985487601820
    }
  ]
}

currently this is generated

type InsertDataPointsRequest struct {
        Items []struct {
                Datapoints []struct {
                        Timestamp int64   `json:"timestamp"`
                        Value     float64 `json:"value"`
                } `json:"datapoints"`
                ID int64 `json:"id"`
        } `json:"items"`
}

But by exploding the nested structs it could look like this

type Datapoint struct {
        Timestamp int64   `json:"timestamp"`
        Value     float64 `json:"value"`
}

type Item struct {
        Datapoints []Datapoint `json:"datapoints"`
        ID int64 `json:"id"`
}

type InsertDataPointsRequest struct {
        Items []item `json:"items"`
}

Which would make the use of the code go from this

	v := InsertDataPointsRequest{
		Items: []struct {
			Datapoints []struct {
				Timestamp int64   `json:"timestamp"`
				Value     float64 `json:"value"`
			} `json:"datapoints"`
			ID int64 `json:"id"`
		}{{
			Datapoints: []struct {
				Timestamp int64   `json:"timestamp"`
				Value     float64 `json:"value"`
			}{
				{
					Timestamp: 12,
					Value:     53,
				},
			},
			ID: 42,
		}},
	}
    
}

to this

	v := InsertDataPointsRequest{
		Items: []Item{{
			Datapoints: []Datapoint{
				{
					Timestamp: 12,
					Value:     53,
				},
			},
			ID: 42,
		},
		},
	}

[Tom Payne]

I would like to implement this. go-xmlstruct already has it (the -named-types flag). However, it's tricky to do in JSON for multiple reasons:

1. JSON only has anonymous objects, so you have to match on property names and types.
2. As properties are optional, you may get unwanted matches.
3. You only have field names to determine type names, which are not necessarily useful.

Consider the following:

{
  "bounds": {
    "topLeft": {
      "x": 1,
      "y": 2
    },
    "bottomRight": {
      "x": 3,
      "y": 4,
      "z": 5
    }
}

The desired Go output would be something like:

package main

type Point struct {
    X int `json:"x"`
    Y int `json:"y"`
    Z int `json:"z,omitempty"`
}

type Bounds struct {
    BottomRight Point `json:"bottomRight"`
    TopLeft     Point `json:"topLeft"`
}

type T struct {
    Bounds Bounds `json:"bounds"`
}

but where would the names Point and Bounds come from? Are the values of topLeft and bottomRight really the same type?

Consider also:

{
    "id": 1,
    "type": "person",
    "name": "Bob",
    "favoriteColor": "red"
}
{
    "id": 2,
    "type": "car",
    "make": "Volkswagen",
    "model": "Rabbit"
}

should these be a single type with name, favoriteColor, make, and model properties? Or separate types?

Fundamentally, I suspect that it's not possible to reliably extract subtypes automatically from JSON. So, when I've encountered cases like this I've manually edited the output of gojsonstruct to match the semantics of the data.

[andoks]

That for sure is an issue. Maybe it could implemented by using a sort of manually specified name map? where the user provides some sort of mapping between objects and typenames for the types to extract and name? Although it comes with a host of potential complexity.

[Tom Payne]

By the time you're writing a manually-specified name map you're starting to define a schema. You can define the schema by writing the Go structs yourself, or editing gojsonstruct's output manually, or using JSON Schema. gojsonstruct is useful when you don't have a schema.

[Tom Payne]

Just to be clear, I think it would be fantastic if gojsonstruct could implement what you're requesting. I'm just not sure how to do it.

[Tom Payne]

Another example, given:

{
    "name": "Alice",
    "friends": {
        "cheshireCat": {
            "magical": true
        }
    }
}

You'd probably want something like:

type Friend struct {
    Magical bool `json:"magical"`
}

type T struct {
    Name    string            `json:"name"`
    Friends map[string]Friend `json:"friends"`
}

But how do you communicate that friends should be a map, not a struct? There's no automatic way to determine this from the property names alone.

Overall, I think what is being requested here is impossible to do automatically. Closing.

[Tom Payne]

For info, if you're using CUE then https://github.com/sivukhin/cuebootstrap has some neat features for controlling the generated schema.