Special characters in the string are incorrectly encoded
peterq opened this issue · comments
peterq commented
test code
func TestStdJsonParseStd(t *testing.T) {
var s = `hello � world`
bin, err := json.Marshal(s)
log.Println(string(bin), err)
var d interface{}
err = json.Unmarshal(bin, &d)
log.Println(d, err)
a := fastjson.Arena{}
bin = a.NewString(s).MarshalTo(nil)
err = fastjson.ValidateBytes(bin)
log.Println(string(bin), err)
}
Output
=== RUN TestStdJsonParseStd
2023/02/20 15:36:09 "hello \u0014 world" <nil>
2023/02/20 15:36:09 hello � world <nil>
2023/02/20 15:36:09 "hello \x14 world" cannot parse JSON: cannot parse string: unknown escape sequence \x; unparsed tail: ""
--- PASS: TestStdJsonParseStd (0.00s)
PASS
Expected:
like the standard library, encode �
to \u0014
insteadof \x14
(which will cuase a problem when decodeing the json string later)
peterq commented
Kostas Papageorgiou commented
Encountered the same issue today. It seems that MarshalTo
generates output that fails to be parsed by Go standard json library.
Kostas Papageorgiou commented
Seems to be the same issue as #86