Lager panics on malformed json with Golang 1.8.3

[Julian Hjortshoj]

Code that used to work in go 1.7 seems to fail now. I am still investigating, but have reproduced the issue in the go playground with the source from models.go

package main

import (
	"encoding/json"
	"fmt"
)

type LogLevel int

const (
	DEBUG LogLevel = iota
	INFO
	ERROR
	FATAL
)

type Data map[string]interface{}

type LogFormat struct {
	Timestamp string   `json:"timestamp"`
	Source    string   `json:"source"`
	Message   string   `json:"message"`
	LogLevel  LogLevel `json:"log_level"`
	Data      Data     `json:"data"`
}

func (log LogFormat) ToJSON() []byte {
	content, err := json.Marshal(log)
	if err != nil {
		if _, ok := err.(*json.UnsupportedTypeError); ok {
			log.Data = map[string]interface{}{"lager serialisation error": err.Error(), "data_dump": fmt.Sprintf("%#v", log.Data)}
			content, err = json.Marshal(log)
		}
		if err != nil {
			panic(err)
		}
	}
	return content
}

func main() {
        badJson := []byte("{this is not json")
	badData := Data{"PlanID": "Existing", "stuffs": json.RawMessage(badJson)}
	d := LogFormat{Timestamp: "a", Source: "b", Message: "c", LogLevel: INFO, Data: badData}
	s := d.ToJSON()
	fmt.Println(string(s))
}

This yields a panic with "json: error calling MarshalJSON for type json.RawMessage: invalid character 't' looking for beginning of object key string"

My guess is that we just need to tweak the recovery code above to accept a broader set of errors. Will comment as/when I know more.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/146122967

The labels on this github issue will be updated when the story is started.

[Julian Hjortshoj]

This change appears to fix the issue. I'm not sure if there is a clever way to make the type check more concise:

func (log LogFormat) ToJSON() []byte {
	content, err := json.Marshal(log)
	if err != nil {
		_, ok1 := err.(*json.UnsupportedTypeError)
		_, ok2 := err.(*json.MarshalerError)
		if ok1 || ok2 {
			log.Data = map[string]interface{}{"lager serialisation error": err.Error(), "data_dump": fmt.Sprintf("%#v", log.Data)}
			content, err = json.Marshal(log)
		}
		if err != nil {
			panic(err)
		}
	}
	return content
}

[Julian Hjortshoj]

Seems like maybe this issue is caused by the encoding/json changes mentioned here but OTOH, I thought it was still working during the brief interval when we were testing against v1.8.1, so perhaps it is an undocumented change from one of the point releases? Go figure...