Using proto_source_root in proto_library breaks <lang>_grpc_[compile|library]

[Haikal Pribadi]

Say I have 2 proto files sitting in //some-package/

• //some-package/A.proto which imports B.proto (without some-package as prefix)
• //some-package/B.proto

Then I compile the proto_library in //some-package/BUILD`, like the following:

proto_library(
    name = "a-proto",
    srcs = ["a-proto"]
    proto_source_root = "some-package"
)
proto_library(
    name = "b-proto",
    srcs = ["b-proto"]
    proto_source_root = "some-package"
)

The above works well because proto_source_root, which allows us to not have to import one proto file into another using the full path relative from WORKSPACE.

Then I do <lang>_grpc_compile, let's say for java_grpc_compile:

load("@org_pubref_rules_proto//java:compile.bzl", "java_grpc_compile")
java_grpc_compile(
    name = "java-proto",
    deps = [":a-proto"],
)

And here it breaks. java_grpc_compile fails to compile with error:

B.proto: file not found.
some-package/A.proto: Import "B.proto" was not found or had errors.

So basically, A.proto cannot find B.proto. However, when I replace the import statement in A.proto with import some-package/B.proto and remove usage of proto_source_root in proto_library everything works, but we really don't want to do this for many reasons.

Does anyone know what could be the cause and how to work around this?

[Haikal Pribadi]

I've isolated the issue on a branch on my repo for you to reproduce it:
https://github.com/haikalpribadi/grakn/tree/bazel-init-pubref-rules-proto

You can find the BUILD file here: https://github.com/haikalpribadi/grakn/blob/bazel-init-pubref-rules-proto/protocol/proto/BUILD

Once you clone the repo, run bazel build //protocol:client-node-proto to reproduce the problem.

[Paul Cody Johnston]

Ah yes the proto_source_root. I'll try to look at this shortly.

[Haikal Pribadi]

Thanks, @pcj !

[Paul Cody Johnston]

Actually after looking into this issue I don't think it is related to proto_source_root. What you are looking for is a new feature that has a PR open that will allow one to modify the import prefix of proto in question.

bazelbuild/bazel#5536

[Haikal Pribadi]

There's also this issue that I opened in grpc-java at about the same time cause i wasn't sure where the cause is. Do you think this is more relevant? grpc/grpc-java#4896

[Paul Cody Johnston]

Ultimately I think the problem reduces to adding additional --proto_path=B.proto=some-package/B.proto arguments to protoc.

It's fairly trivial to add that feature to all the proto_compile rules, but since we are dependent on the native proto_library rule, unless that implementation has the ability to communicate those arguments to protoc itself (since it calls protoc to generate the descriptor_set), it won't work.

Another option is to rewrite the proto_library in skylark using aspects AND conform to the same provider structure that the native rule returns. I'm a little tempted to try this.

[Paul Cody Johnston]

Or, redo your protos to have a common import structure, like the googleapis repo. But for many organizations with existing build workflow constraints and dependent code, it's not practical to do this.

[Haikal Pribadi]

Well, proto_library can already handle this, by taking in proto_source_root argument. Can this be extended/used in the {lang}_grpc_compile rules?

[Paul Cody Johnston]

Are you able to build the proto_library target in this case (separate from any {lang}_grpc_compile rule)?

Does bazel build //protocol:client-proto figure out the imports and compile the descriptor set correctly?

[Haikal Pribadi]

Yes, proto_library compiles correctly with proto_source_root, since Bazel 0.17.2. So "yes" to both the above questions, @pcj :)