ancient-rose-27306
08/19/2022, 6:12 AM./pants tailor
and got following output:
(pants) [root@9a5752857756 pants-training]# ./pants tailor
Created src/client/BUILD:
- Add python_sources target client
- Add pex_binary target greeter_client
Created src/proto/BUILD:
- Add protobuf_sources target proto
Created src/server/BUILD:
- Add pex_binary target greeter_server
- Add python_requirements target reqs
- Add python_sources target server
(pants) [root@9a5752857756 pants-training]#
After that I did ./pants package src/server/greeter_server.py
and encountered following error:
(pants) [root@9a5752857756 pants-training]# ./pants package src/server/greeter_server.py
05:54:49.45 [ERROR] 1 Exception encountered:
IndexError: string index out of range
(pants) [root@9a5752857756 pants-training]#
Could someone please help? My goal is to generate the stubs from the proto files and publish them as python wheels so that I can pip install them and use them in client and server code.
Following is how my repo looks:
(pants) [root@9a5752857756 pants-training]# /tree.sh
Tree of: /root/pants-training
.
|__pants
|__src
| |__proto
| | |__helloworld.proto
| | |__BUILD
| |__server
| | |__requirements.txt
| | |__greeter_server.py
| | |__BUILD
| |__client
| | |__greeter_client.py
| | |__BUILD
|__.pants.d
| |__exceptions.log
| |__exceptions.471.log
| |__run-tracker
| | |__pants_run_2022_08_19_05_55_17_757_5184323f0cce42809255925d6987ccbe
| | | |__logs
| | |__pants_run_2022_08_19_05_55_11_114_e3698008d5454c2aa6d584bf5680eaed
| | | |__logs
| | |__pants_run_2022_08_19_05_53_24_325_48f808a97d1c4c8aa6faad9261780e60
| | | |__logs
| | |__pants_run_2022_08_19_05_48_46_160_81dfa93f029b44f0bc191a8560c255c6
| | | |__logs
| | |__pants_run_2022_08_19_05_54_45_113_dee6bff2c02a414c94a754c60781580c
| | | |__logs
| | |__pants_run_2022_08_19_05_54_08_319_a3964cb6faa049c7917933ccfe3ede11
| | | |__logs
| |__pants.log
|__pants.toml
|__.pids
| |__855f83c10083
| | |__pantsd
| | | |__socket
| | | |__pid
| | | |__process_name
| | | |__fingerprint
| |__.lock.pantsd
refined-addition-53644
08/19/2022, 6:39 AMpython_distribution
inside the BUILD file and then run `./pants package src/server`:
https://www.pantsbuild.org/docs/python-distributions#the-python_distribution-targetancient-rose-27306
08/19/2022, 6:59 AMpython_distribution
target for them and then package them as a wheel.refined-addition-53644
08/19/2022, 7:04 AMpython_distribution
does. In dependencies
you need to add the dependency on underlying source code. In this case you can use the
python_sources
target called server
which I guessed from this
Created src/server/BUILD:
- Add pex_binary target greeter_server
- Add python_requirements target reqs
- Add python_sources target server
So dependencies=[":server"]
ancient-rose-27306
08/19/2022, 7:09 AMsrc/server/BUILD
?refined-addition-53644
08/19/2022, 7:09 AMserver
ancient-rose-27306
08/19/2022, 7:12 AMsrc/proto/helloworld.proto
. The code isn't present here in the repo because it is not generated yet. I am trying to figure out how to generate that using pants and that's where I need help.refined-addition-53644
08/19/2022, 7:19 AM./pants export-codegen ::
ancient-rose-27306
08/19/2022, 7:22 AM./pants export-codegen ::
but ran into some error.refined-addition-53644
08/19/2022, 7:23 AMexport-codegen
and make sure it runs successfully before moving to packagingpython_distribution
or pex_binary
, what pants does in background is
1. Run export-codegen (my guess)
2. package the export-codegen generated output with these goalsPants will not actually write the generated files to disk, except when running. Instead, any targets that depend on the protocol targets will cause their code to be generated, and those generated files will be copied over into the “chroot” (temporary directory) where Pants executes../pants export-codegen
ancient-rose-27306
08/19/2022, 7:33 AMrefined-addition-53644
08/19/2022, 7:35 AM./pants publish
ancient-rose-27306
08/19/2022, 7:38 AMrefined-addition-53644
08/19/2022, 7:40 AMdependencies=[:server]
in your python_distribution
ancient-rose-27306
08/19/2022, 7:46 AMdependencies=[:server]
in server's own BUILD file src/server/BUILD
. Server is dependent on proto.refined-addition-53644
08/19/2022, 8:06 AMancient-rose-27306
08/19/2022, 1:53 PM./pants package src/server:grpc-server
and get this error. Do you have any idea?happy-kitchen-89482
08/19/2022, 2:19 PMOk, so does it mean pants will auto generate at run time based on import statement?Correct!
export-codegen
is useful for debugging, but not needed in normal usage.dependencies=
) from the target using the stubs to the protobuf_sources
target owning the .proto filesIndexError: string index out of range
is a very bad error message of courseimport
statements. The one case where it cannot is a python_distribution
must have an explicit dependency at least on its entry point (usually Pants can infer the entry point’s dependencies, and so on)ancient-rose-27306
08/19/2022, 2:25 PMAnother useful thing for debugging is to add an explicit dependency (inIf I understand the above statement correctly, then I must add the dependencies of both client and server in the BUILD file of proto (assuming you are familiar with the repo structure I posted yesterday).) from the target using the stubs to thedependencies=
target owning the .proto filesprotobuf_sources
Can you bundle up the code that reproduces this error, and push it to github? Then I can debug easilyOn it way!
happy-kitchen-89482
08/19/2022, 2:29 PMIf I understand the above statement correctly, then I must add the dependencies of both client and server in the BUILD file of proto (assuming you are familiar with the repo structure I posted yesterday).Well, both the client and the server should be inferring the dependency on the proto, we just need to debug why they are not
ancient-rose-27306
08/19/2022, 2:30 PMPants usually infers dependencies based onI have a question from this repo for import statements. The python codestatements. The one case where it cannot is aimport
must have an explicit dependency at least on its entry point (usually Pants can infer the entry point’s dependencies, and so on)python_distribution
simple_example_test.py
in src/python/protobuf_examples
imports the stubs from simple_example.v1.person_pb2
. But according to this BUILD file, the stubs should be generating in src/python
.And note that the dependency would go in the BUILD file of the server and the BUILD file of the client, on the proto, not in the BUILD file of the protoYeah, so for now I have just added the dependency in server. Let me push the code real quick and come back to the messages later.
happy-kitchen-89482
08/19/2022, 2:34 PMsrc/python/foo/bar.py
is imported as foo.bar
src
or src/python
(pants knows about these automatically). Or sometimes they are just a directory named after the project, in which case you have to tell Pants about them.ancient-rose-27306
08/19/2022, 3:29 PM.
src
src/protobuf
src/python
src/thrift
The source roots and everything made sense but then following comment and code line confused me:
# This will generate files under `src/python`, rather than `src/protobuf`, which is convenient
# so that we do not need to add `__init__.py` files to `src/protobuf`. See
# <https://www.pantsbuild.org/docs/protobuf-python#protobuf-and-source-roots>
python_source_root="src/python",
If the code is being generated in src/python then why are we importing it from simple_example.v1
?