[protobuf-go]: bug when compile protobuf directly from proto file to protoregistry

Question

[protobuf-go]: bug when compile protobuf directly from proto file to protoregistry

Sivys opened this issue 8 months ago · comments

What version of protobuf and what language are you using?
Version:
protobuf version: google.golang.org/protobuf@v1.31.0
golang version: all version

What did you do?
Here is my code, I want directly compile and register protobuf from a proto file to protoregistry

package proto_test

import (
    "strings"
    "testing"

    "github.com/bufbuild/protocompile/parser"
    "github.com/bufbuild/protocompile/reporter"
    "google.golang.org/protobuf/reflect/protodesc"
    "google.golang.org/protobuf/reflect/protoregistry"
)

const protofile = `
syntax = "proto3";

package test;

message Foo {
    map<int32, Bar> Bar = 1; // i know it's not good name define
}
  
message Bar {
    int32 integer = 1;
}
`

func TestParseFile(t *testing.T) {
    testParseProtoFile("test.proto")
}

func testParseProtoFile(filename string) {
    errHandler := reporter.NewHandler(nil)
    ast, err := parser.Parse(filename, strings.NewReader(protofile), errHandler)
    if err != nil {
        panic(err)
    }

    result, err := parser.ResultFromAST(ast, true, errHandler)
    if err != nil {
        panic(err)
    }

    fdp := result.FileDescriptorProto()
    _, err = protodesc.NewFile(fdp, protoregistry.GlobalFiles)
    if err != nil {
        panic(err)
    }
    // ...
}

but the NewFile returns an error

proto: message field "test.Foo.BarEntry.value" cannot resolve type: unknown kind

so I try to find out what's going on, finally find out that at [HERE](https://github.com/protocolbuffers/protobuf-go/blob/8088bf85b8fc5a6c0d7e2a39743e237cae8bef99/reflect/protodesc/desc_resolve.go#L126)

d, err := r.findDescriptor(scope, ref)
...
switch d := d.(type) {
case protoreflect.EnumDescriptor:
	return protoreflect.EnumKind, d, nil, nil
case protoreflect.MessageDescriptor:
	return protoreflect.MessageKind, nil, d, nil
default:
	return 0, nil, nil, errors.New("unknown kind")
}

the type of d is filedesc.Field which I think it should be filedesc.Message

so I printed out all descriptors that registered into descsByName at [HERE](https://github.com/protocolbuffers/protobuf-go/blob/8088bf85b8fc5a6c0d7e2a39743e237cae8bef99/reflect/protodesc/desc_init.go#L236), it shows

test.Foo.Bar
test.Foo.BarEntry
test.Foo.BarEntry.key
test.Foo.BarEntry.value
test.Bar
test.Bar.integer
test.Foo

the [function](https://github.com/protocolbuffers/protobuf-go/blob/8088bf85b8fc5a6c0d7e2a39743e237cae8bef99/reflect/protodesc/desc_resolve.go#L161) findDescriptor will search descriptors from bottom to top, in this case it will stop at test.Foo.Bar not test.Bar which caused the error above

What did you expect to see?

I know it's bad to name a field like that, since my syntax is valid in proto file and the file can be processed by protoc correctly and generated the right pb code, so I think the protobuf lib should performance the same too?

What did you see instead?

It can not process the file correctly and returns me an error

Anything else we should know about your project / environment?

no

Damien Neil · Answer 1 · Thu Sep 28 2023 00:29:27 GMT+0800 (China Standard Time)

This looks like a bug in "github.com/bufbuild/protocompile/parser".

If I parse your .proto file with protoc, I get:

name:  "test.proto"
package:  "test"
message_type:  {
  name:  "Foo"
  field:  {
    name:  "Bar"
    number:  1
    label:  LABEL_REPEATED
    type:  TYPE_MESSAGE
    type_name:  ".test.Foo.BarEntry"
    json_name:  "Bar"
  }
  nested_type:  {
    name:  "BarEntry"
    field:  {
      name:  "key"
      number:  1
      label:  LABEL_OPTIONAL
      type:  TYPE_INT32
      json_name:  "key"
    }
    field:  {
      name:  "value"
      number:  2
      label:  LABEL_OPTIONAL
      type:  TYPE_MESSAGE
      type_name:  ".test.Bar"
      json_name:  "value"
    }
    options:  {
      map_entry:  true
    }
  }
}
message_type:  {
  name:  "Bar"
  field:  {
    name:  "integer"
    number:  1
    label:  LABEL_OPTIONAL
    type:  TYPE_INT32
    json_name:  "integer"
  }
}
syntax:  "proto3"

If I parse it with the bufbuild parser:

name:  "test.proto"
package:  "test"
message_type:  {
  name:  "Foo"
  field:  {
    name:  "Bar"
    number:  1
    label:  LABEL_REPEATED
    type_name:  "BarEntry"
    json_name:  "Bar"
  }
  nested_type:  {
    name:  "BarEntry"
    field:  {
      name:  "key"
      number:  1
      label:  LABEL_OPTIONAL
      type:  TYPE_INT32
      json_name:  "key"
    }
    field:  {
      name:  "value"
      number:  2
      label:  LABEL_OPTIONAL
      type_name:  "Bar"
      json_name:  "value"
    }
    options:  {
      map_entry:  true
    }
  }
}
message_type:  {
  name:  "Bar"
  field:  {
    name:  "integer"
    number:  1
    label:  LABEL_OPTIONAL
    type:  TYPE_INT32
    json_name:  "integer"
  }
}
syntax:  "proto3"

The message-valued fields are missing type: TYPE_MESSAGE, and (the source of your error) do not use fully qualified names in the type_name field.

Sivys · Answer 2 · Thu Sep 28 2023 10:04:54 GMT+0800 (China Standard Time)

This looks like a bug in "github.com/bufbuild/protocompile/parser".

If I parse your .proto file with protoc, I get:

name:  "test.proto"
package:  "test"
message_type:  {
  name:  "Foo"
  field:  {
    name:  "Bar"
    number:  1
    label:  LABEL_REPEATED
    type:  TYPE_MESSAGE
    type_name:  ".test.Foo.BarEntry"
    json_name:  "Bar"
  }
  nested_type:  {
    name:  "BarEntry"
    field:  {
      name:  "key"
      number:  1
      label:  LABEL_OPTIONAL
      type:  TYPE_INT32
      json_name:  "key"
    }
    field:  {
      name:  "value"
      number:  2
      label:  LABEL_OPTIONAL
      type:  TYPE_MESSAGE
      type_name:  ".test.Bar"
      json_name:  "value"
    }
    options:  {
      map_entry:  true
    }
  }
}
message_type:  {
  name:  "Bar"
  field:  {
    name:  "integer"
    number:  1
    label:  LABEL_OPTIONAL
    type:  TYPE_INT32
    json_name:  "integer"
  }
}
syntax:  "proto3"

If I parse it with the bufbuild parser:

name:  "test.proto"
package:  "test"
message_type:  {
  name:  "Foo"
  field:  {
    name:  "Bar"
    number:  1
    label:  LABEL_REPEATED
    type_name:  "BarEntry"
    json_name:  "Bar"
  }
  nested_type:  {
    name:  "BarEntry"
    field:  {
      name:  "key"
      number:  1
      label:  LABEL_OPTIONAL
      type:  TYPE_INT32
      json_name:  "key"
    }
    field:  {
      name:  "value"
      number:  2
      label:  LABEL_OPTIONAL
      type_name:  "Bar"
      json_name:  "value"
    }
    options:  {
      map_entry:  true
    }
  }
}
message_type:  {
  name:  "Bar"
  field:  {
    name:  "integer"
    number:  1
    label:  LABEL_OPTIONAL
    type:  TYPE_INT32
    json_name:  "integer"
  }
}
syntax:  "proto3"

The message-valued fields are missing type: TYPE_MESSAGE, and (the source of your error) do not use fully qualified names in the type_name field.

my bad, i thought github.com/bufbuild/protocompile/parser is a official package