https://pantsbuild.org/ logo
Powered by
f

freezing-quill-50372

03/09/2022, 1:52 AM
Hello, how do I generate protobuf’s python codes into the specific python root directory ? I already specified 
python_source_root
, but the codes are generated in the original directory (detailed settings are following in thread)
Copy code
├── apis
│   ├── README.md
│   └── protos
│       ├── BUILD
│       ├── README.md
│       └── some_proto.proto
├── python
│   ├── README.md
│   ├── pkg
│   │   ├── README.md
│   │   ├── apis
│   │   │   ├── BUILD
│   │   │   ├── README.md
│   │   │   ├── common_namespace
│   │   │   │   ├── README.md
│   │   │   │   └── apis
│   │   │   │       ├── __init__.py
│   │   │   │       ├── exceptions.py
│   │   │   │       └── protos
│   │   │   │           └── __init__.py <- I would like to generate the codes into this directory
│   │   │   ├── poetry.lock
│   │   │   ├── pyproject.toml
│   │   │   └── tests
│   │   │       └── __init__.py
...
│   ├── projects
│   │   ├── README.md
│   │   ├── some_project
│   │   │   ├── BUILD
│   │   │   ├── README.md
│   │   │   ├── poetry.lock
│   │   │   ├── pyproject.toml
│   │   │   ├── some_project
│   │   │   │   ├── __init__.py
│   │   │   │   ├── interface
│   │   │   │   │   ├── __init__.py
│   │   │   │   │   ├── cmd
│   │   │   │   │   │   └── __init__.py
│   │   │   │   │   └── grpc
│   │   │   │   │       ├── __init__.py
│   │   │   │   │       ├── main.py
│   │   │   │   │       └── servicier.py
...
BUILD
in the 
apis/protos
is following.
Copy code
protobuf_sources(
    name="protos",
    grpc=True,
    python_source_root="python/pkg/apis/common_namespace"
)
some_project
’s 
BUILD
refers the protobuf target as following.
Copy code
python_sources(
    name="pkg",
    sources=["some_project/**/*.py"],
    dependencies=[
        "apis/protos:protos",
        "python/pkg/utils:pkg",
        "python/pkg/clients:pkg",
        "python/pkg/apis:pkg",
        ":pydantic",
        ":grpcio",
        ":protobuf",
    ],
)
h

hundreds-father-404

03/09/2022, 2:03 AM
Hello! can you please run 
./pants roots
and share it?
Also are you running 
./pants export-codegen
to see where it's being generated into?
f

freezing-quill-50372

03/09/2022, 2:04 AM
Thanks! I’ll try them
./pants root
result is here,
Copy code
python/pkg/apis
python/projects/some_project
./pants export-codegen
’s result is here. It is odd, protobuf python codes themselves are generated when I packaged the 
some_project
…
Copy code
02:06:14.36 [WARN] No codegen files/targets matched. All codegen target types: docker_image, protobuf_source
h

hundreds-father-404

03/09/2022, 2:10 AM
Oh to clarify, 
./pants export-codegen ::
. You have to tell Pants what to run on
https://pantsbuild.slack.com/archives/C046T6T9U/p1646791558241399?thread_ts=1646790753.551439&amp;cid=C046T6T9U Okay great. So, 
python_source_root
needs to be set to one of those specifically. Without the extra folders at the end
f

freezing-quill-50372

03/09/2022, 2:14 AM
By the way, 
./pants export-codegen ::
said following. The following results seem to lack the namespace 
some_namespace
.
Copy code
└── python
    ├── pkg
    │   └── apis
    │       └── apis
    │           └── protos
    │               ├── some_proto_pb2.py
    │               └── some_proto_pb2_grpc.py
Then, Pants ignore implicit namespaces ?
h

hundreds-father-404

03/09/2022, 9:47 PM
That's right. Protobuf files are generated using their file path, plus some source root manipulation. Do you have with 
./pants roots
an entry like 
/
btw? If not, I'm surprised Pants is not erroring. Assuming that you have the source root 
/
in 
./pants roots
, then here is what's happening to get that path. 1. You have the file 
api/protos/some_protos.proto
- it matches the source root 
/
, which means there is no prefix to strip. So you stick with 
api/protos/some_protos.proto
, which then generates the file 
api/protos/some_protos_pb2.py
2. Because you are setting the field 
python_source_root
, that gets added by Pants as a prefix. So you end up with 
python/pkg/apis/api/protos/some_protos.proto
If you want to generate into a different directory, then you need to locate the 
.proto
file in a different location like 
common_namespace/apis/protos/some_proto.proto
Does that make sense?
Then, Pants ignore implicit namespaces ?
What do you mean? How are you doing this currently without Pants?
f

freezing-quill-50372

03/09/2022, 10:32 PM
Thanks for your detailed explanation, I got the process of proto codes generation.
What do you mean? How are you doing this currently without Pants?
Never mind, actually it seems to be my misunderstand. I’m building project structure from scratch for a new product and struggling how to confirm it.
❤️ 1
h

hundreds-father-404

03/09/2022, 10:58 PM
Ah okay, cool! Btw have you seen https://github.com/pantsbuild/example-codegen?
Powered by