valyala / fastjson

Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Special characters in the string are incorrectly encoded

peterq opened this issue · comments

commented

test code

func TestStdJsonParseStd(t *testing.T) {
	var s = `hello � world`

	bin, err := json.Marshal(s)
	log.Println(string(bin), err)

	var d interface{}
	err = json.Unmarshal(bin, &d)
	log.Println(d, err)

	a := fastjson.Arena{}
	bin = a.NewString(s).MarshalTo(nil)
	err = fastjson.ValidateBytes(bin)
	log.Println(string(bin), err)
}

Output

=== RUN   TestStdJsonParseStd
2023/02/20 15:36:09 "hello \u0014 world" <nil>
2023/02/20 15:36:09 hello � world <nil>
2023/02/20 15:36:09 "hello \x14 world" cannot parse JSON: cannot parse string: unknown escape sequence \x; unparsed tail: ""
--- PASS: TestStdJsonParseStd (0.00s)
PASS

Expected:
like the standard library, encode to \u0014 insteadof \x14 (which will cuase a problem when decodeing the json string later)

Encountered the same issue today. It seems that MarshalTo generates output that fails to be parsed by Go standard json library.

Seems to be the same issue as #86