MarshalText incorrectly handles unknown bytes

[Dmitry Vyukov]

The following program crashes with the following message:

package main

import (
    "bytes"
    "fmt"

    pb "github.com/dvyukov/go-fuzz/examples/protobuf/pb"
    "github.com/golang/protobuf/proto"
)

func main() {
    data := []byte("\n\x010")
    v := new(pb.M25)
    err := proto.Unmarshal(data, v)
    if err != nil {
        panic(err)
    }

    var buf bytes.Buffer
    err = proto.MarshalText(&buf, v)
    if err != nil {
        panic(err)
    }
    v2 := new(pb.M25)
    err = proto.UnmarshalText(buf.String(), v2)
    if err != nil {
        fmt.Printf("failed to UnmarshalText: %q\n", buf.Bytes())
        panic(err)
    }
}

failed to UnmarshalText: "/* 3 unknown bytes */\n1: \"0\"\n"
panic: line 1.0: unexpected byte 0x2f

MarshalText should not produce text that is unparsable by UnmarshalText.
Either MarshalText should write unknown bytes differently, or UnmarshalText accept the current output of MarshalText.

on commit 34a5f24

[David Symonds]

No, the text format is not an interchange format. There is no guarantee that MarshalText's output will be parseable by UnmarshalText, especially when it fails.

[Dmitry Vyukov]

MarshalText does not fail in any way detectable by user.
Do you mean that the "unknown bytes" are not representable in text form?

[David Symonds]

There are quite a few ways that a proto struct might not be representable in text format; having unrecognised fields is one way, and that's what your example hit.