Getting errors when i add parametrize to my protobuf sources

protobuf_sources(
    **parametrize("py39", python_interpreter_constraints=[">=3.9.15","<3.10"], python_resolve="default"),
    **parametrize("py310", python_interpreter_constraints=["==3.10.*"], python_resolve="other"),
)

I get this error when i run

export-codegen

22:04:43.84 [ERROR] 1 Exception encountered:

Engine traceback:
  in `export-codegen` goal

ProcessExecutionFailure: Process 'Generating Go sources from path/to/proto@parametrize=py39.' failed with exit code 1.
stdout:

stderr:
path/to/proto/a.proto: File not found.
path/to/proto/b.proto:6:1: Import "path/to/proto/a.proto" was not found or had errors

Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

The folder has a

BUILD.pants

and a number of protos that import from each other, worked before adding parametrize.

Huh. I wonder if this is an issue with codegen and parametrizations. Are you able to share an example repo that reproduces this error?

@curved-television-6568 sorry for the slow reply, I've recreated the bug here https://github.com/hardishw/pants-parametrize

Ah, sorry, I completely blanked about this. Took a quick look now. I think the preceding warning might be informative about what the root issue could be here:

220835.45 [WARN] The target src/protos/test.proto@parametrize=py39 imports

protos/testtwo.proto

in the file src/protos/test.proto, but Pants cannot safely infer a dependency because more than one target owns this file, so it is ambiguous which to use: ['src/protos/testtwo.proto@parametrize=py310', 'src/protos/testtwo.proto@parametrize=py39'].

Please explicitly include the dependency you want in the

dependencies

field of src/protos/test.proto@parametrize=py39, or ignore the ones you do not want by prefixing with

!

or

!!

so that one or no targets are left.

Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this file. Refer to https://www.pantsbuild.org/2.21/docs/using-pants/troubleshooting-common-issues#import-errors-and-missing-dependencies.

Huh. Seems dep inference doesn't handle this situation very well, as indicated by the warning message. I explicitly added the dependency and then the export worked. Def worth filing an issue for, you have the great repro case to go with it 👍

❯ git diff
diff --git a/src/protos/BUILD.pants b/src/protos/BUILD.pants
index 01ca744..580fe83 100644
--- a/src/protos/BUILD.pants
+++ b/src/protos/BUILD.pants
@@ -1,4 +1,9 @@
 protobuf_sources(
+    overrides={
+        'test.proto': dict(
+            dependencies=['src/protos/testtwo.proto'],
+        ),
+    },
     **parametrize("py39", python_interpreter_constraints=[">=3.9.15","<3.10"]),
-    **parametrize("py310", python_interpreter_constraints=["==3.10.*"]),
+    **parametrize("py310", python_interpreter_constraints=["==3.10.*"], python_resolve="object_detection"),
 )

(I had thrown in the resolve there for good measure, but not sure if it was the right thing to do, nor needed..)

❯ tree dist
dist
└── codegen
    └── src
        └── protos
            ├── test.pb.go
            ├── test_pb2.py
            ├── test_pb2.pyi
            ├── testtwo.pb.go
            ├── testtwo_pb2.py
            └── testtwo_pb2.pyi

4 directories, 6 files

I think I'm seeing this same issue. The proto file automatic dependency resolving fails to handle the parameterized fields (since multiple targets own the file). In my case, it's the

python_resolve

field that's parameterized. If I explicitly include the generic proto dependency, it resolves everything fine. But this isn't a great solution for me as we have so many proto files interconnecting. It's annoying to manually handle all the dependencies.