what's the reasoning behind omitempty working differently from encoding/json?

[Yan-Fa Li]

Hi, love this library, thank you so much.

Question I was recently caught out by the way omit empty works in structs tags. For encoding/json if you pass a pointer it omits the field, but if you pass structs a nil it sets the value to null. It took me a bit to understand that structs was detecting uninitialized fields to decide whether to omit empty. Is there a rational behind this different from the encoding/json? Thanks

[Fatih Arslan]

Hi @yanfali

Can you give a concrete example for both encoding/json and fatih/structs on how what you see and on what you want to see ?

[Yan-Fa Li]

Hi @fatih, I have finally figured out what is going on. So it's a bit of a corner case, and my original description of the problem is flawed. So the issue seems to be, if I have a pointer to an initialized but un-used struct and I encode it to a map type then I take the map and I encode it using the standard JSON encoder it doesn't seem to honor the omitempty struct tags and instead falls back to the default JSON encoding for the base object.

If this is just an incorrect usage of the API I can live with that, but I thought I would ask if it was a bug or working as intended. Thanks again.

Here's a test case which illustrates the problem:

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"github.com/fatih/structs"
	"testing"
)

type plain2 struct {
	Name             string  `structs:"name,omitempty"`
	OptionalNickName *string `structs:"nickName,omitempty"`
}

type plain3 struct {
	Person *plain2 `structs:"person,omitempty"`
}

func TestJSONEncodingMapForUnusedStructIsIncorrect(t *testing.T) {
	var person plain2
	src := plain3{Person: &person}
	val := structs.Map(&src)
	fmt.Printf("%+v\n", val)

	buf := bytes.NewBuffer([]byte{})
	enc := json.NewEncoder(buf)
	err := enc.Encode(&val)
	if err != nil {
		t.Fatal("encoding failed ", err)
	}
	// This one generates valid JSON but ignores the omit empty
	// for pointer plain2 and doesn't honor the structs tags for mapping the field names
	fmt.Printf("%+v\n", buf.String())
}

func TestJSONEncodingMapForUsedStructIsCorrect(t *testing.T) {
	person := plain2{Name: "John"}
	src := plain3{Person: &person}
	val := structs.Map(&src)
	fmt.Printf("%+v\n", val)

	buf := bytes.NewBuffer([]byte{})
	enc := json.NewEncoder(buf)
	err := enc.Encode(&val)
	if err != nil {
		t.Fatal("encoding failed ", err)
	}
	// this one has a single field set within the pointed to object
	// but does honor the field names and omit empty
	fmt.Printf("%+v\n", buf.String())
}

Here's the output:

go test -v
=== RUN   TestJSONEncodingMapForUnusedStructIsIncorrect
map[person:0xc4200a0000]
{"person":{"Name":"","OptionalNickName":null}}

--- PASS: TestJSONEncodingMapForUnusedStructIsIncorrect (0.00s)
=== RUN   TestJSONEncodingMapForUsedStructIsCorrect
map[person:map[name:John]]
{"person":{"name":"John"}}

--- PASS: TestJSONEncodingMapForUsedStructIsCorrect (0.00s)
PASS
ok  	github.com/yanfali/struct	0.007s

[Yan-Fa Li]

p.s. I'm running on commit dc3312c - a vendored version of structs.

[Fatih Arslan]

Hi @yanfali

So I've checked both examples. The first one is actually no bug and is indented. It actually goes over Name and OptionalNickName and because they both are empty it skips them indeed. But the person variable (of type plain2) you're passing is not nil. It's a pointer of person. Therefore it just adds it there without changing it, so you get this: map[person:0xc4200a0000]. And because it's not nil, json encodes it to: {"person":{"Name":"","OptionalNickName":null}} which is correct. So it's correct in both json and structs.

The second example is also correct because this time you're passing a modified person (person := plain2{Name: "John"}), thus the structs package correctly replaces it and you'll get: map[person:map[name:John]] which the json package then encodes to {"person":{"name":"John"}}

If you set person as nil the structs package indeed skips it and you'll end up with an empty map.
Let me know if this is not clear.

If not can you share what you're expecting in the first example? I'll go over it again. Thanks

[Yan-Fa Li]

Hi @fatih I'm a little confused by this behavior. I would have expected structs to descend into person in the first example as it did in the 2nd one and discover that there we no fields and simply leave person as nil. struct understands enough to descend into the 2nd example and only map the field that is actually required so I find this surprising behavior for the Map function.

[Yan-Fa Li]

How I fixed it in actual code was to stop using pointers with structs.Map. When using an instance it works as expected when the fields are default. I will see what the json encoder does in the same situation and report back.