Bug: google/protobuf/go_features.proto: File not found.

[chrisprobst]

What version of protobuf and what language are you using?
Version: v1.36.0

What did you do?

// demo.proto
edition = "2023";

import "google/protobuf/go_features.proto";

option features.(pb.go).api_level = API_OPAQUE;

message Demo {
  string name = 1;
}

protoc --go_out=. --go-grpc_out=. demo.proto

Following the recent Go blog post and tutorial about OPAQUE API.

What did you expect to see?
The compiled files.

What did you see instead?

$ protoc --go_out=. --go-grpc_out=. demo.proto
google/protobuf/go_features.proto: File not found.
demo.proto:3:1: Import "google/protobuf/go_features.proto" was not found or had errors.

I checked the docs twice, I can't find additional information about this. Is this a bug?

[stevvooe]

I've found the source of the file in https://github.com/protocolbuffers/protobuf/blob/main/go/go_features.proto but its not clear to me how this should be distributed.

You can drop it into a google/protobuf directory to get going but I'm not how scalable that will be. Something like this:

❯ mkdir -p google/protobuf
❯ curl -sL https://raw.githubusercontent.com/protocolbuffers/protobuf/refs/heads/main/go/go_features.proto > google/protobuf/go_features.proto
❯ protoc -I. --proto_path=proto --go_out=gen --go_opt=paths=source_relative proto/myproto.proto

Would be nice if this was a part of the primary distribution, since you'll need these feature files for all languages to generate protobufs. This has been a long term pain point.

[stevvooe]

I got this working with buf fairly well:

1. Setup buf.yaml with buf config init and add this line:

deps:
  - buf.build/protocolbuffers/gofeatures

That declares a dependency on that buf package.
2. Run buf dep update and it will pull the dependency down. (Doesn't work without this)
3. Then, in buf.gen.yaml, you can do things like normal. In this example, I've enabled API_OPQAUE:

plugins:
  - remote: buf.build/protocolbuffers/go
    out: gen
    opt:
      - paths=source_relative
      - default_api_level=API_OPAQUE

4. Run buf generate to create the Go files.

If you run into issues, make sure that buf, protoc and protoc-gen-go are all the latest version. I've left out some setup items but you can check out https://buf.build/docs/ if you get stuck.

[chrisprobst]

Thanks for looking into this. The idea of copying the file works, however, I thought protoc + plugins should pickup those builtin files automatically.

Regarding buf: Indeed this works fine and I agree that buf builds a nice abstraction. I have to ask as I am not using buf so far and only protobuf directly. Is buf the recommended approach? It seems to me as a complete alternative implementation with nice tooling around it.

In my case, I have to use Google's implementation and my feeling teels me this is a bug as this should work. But thanks for the workaround of copying the file manually, good idea!

[chrisprobst]

Just one more observation: When I add

--go_opt=default_api_level=API_OPAQUE

to the protoc command, it works. So, I think the problem is specifying the API level inside a proto file:

import "google/protobuf/go_features.proto";
option features.(pb.go).api_level = API_OPAQUE;

[stevvooe]

I think the problem is specifying the API level inside a proto file

Yes, that seems to be the fundamental issue. If you aren't using buf, I'd just check in a protodeps directory that gets included or just have it next to your regular protos:

protos/
   google/protobuf/go_features.proto
   yourapi/v1/....

Then, I think you can exec like this:

protoc --proto_path=protos --go_out=gen --go_opt=paths=source_relative yourapi/v1/....

You may need an -I./protos but I haven't tried that out.

[stapelberg]

Sorry for not seeing this issue earlier.

Following the recent Go blog post and tutorial about OPAQUE API.

I checked the docs twice, I can't find additional information about this. Is this a bug?

If the google/protobuf/go_features.proto file is not found, that means your protoc version is too old (or your installation is faulty somehow). As the docs mention, you need at least version 29.0, which includes the go_features.proto file as a well-known proto.

[chrisprobst]

I use latest protoc from homebrew, it’s 29.2. I gave up by now. I think I wait until the packages are more stable around the Opaque API. I tried many different things, only buf was able to compile it using their special gofeatures dependency. In case you have some time I would appreciate of you could try this in small repo and see if this really works for you. I can‘t see how this can work without manually copying the file from the repo. Even the most simple proto file with latest protoc this error is raised. In case this works for you, maybe you can share some Details Like the exact protoc flags.

[meling]

Looks like this is a homebrew issue. I downloaded the official release from here: https://github.com/protocolbuffers/protobuf/releases/tag/v29.2, and it contains:

% fd go_features.proto
include/google/protobuf/go_features.proto

However, checking the homebrew package, it isn't included:

% pwd
/opt/homebrew/Cellar/protobuf/29.2
% fd go_features.proto
# Finds nothing

I can confirmed that the official release can compile the example you gave in the first report with a slight tweak:

option go_package = "example.com/fizz";

% protoc --go_out=. --go-grpc_out=. demo.proto
% tree example.com/
example.com/
└── fizz
    └── demo.pb.go

2 directories, 1 file

[lstoll]

Looks like there's a problem with the cmake build, which homebrew is using. There's a proposed fix in protocolbuffers/protobuf#19773

I'm assuming the official release is using bazel, which doesn't have this issue. Once this lands and homebrew updates it should be good.

[chrisprobst]

Thanks for double-checking on this! Highly appreciated!