ancient-rose-27306
08/19/2022, 2:21 PMIndexError: string index out of range
is a rather generic one. See example below:
(pants) [root@9a5752857756 pants-training]# ./pants dependencies src/server/greeter_server.py
src/proto/helloworld.proto
src/server/greeter_server.py
(pants) [root@9a5752857756 pants-training]# ./pants dependees src/server/greeter_server.py
14:18:47.17 [ERROR] 1 Exception encountered:
IndexError: string index out of range
(pants) [root@9a5752857756 pants-training]#
The dependencies
goal worked but dependees
failed on the same target. If the target doesn’t have any dependees then I guess it shouldn’t fail.happy-kitchen-89482
08/19/2022, 2:26 PMhundreds-father-404
08/19/2022, 2:32 PM--print-stacktrace
would help, pleaseancient-rose-27306
08/19/2022, 3:31 PMhundreds-father-404
08/19/2022, 3:49 PMancient-rose-27306
08/19/2022, 5:21 PM(pants) [root@9a5752857756 pants-training]# ./pants --print-stacktrace package src/server:grpc-server
17:20:05.42 [ERROR] 1 Exception encountered:
Engine traceback:
in select
in pants.core.goals.package.package_asset
in pants.backend.python.goals.setup_py.package_python_dist (src/server:grpc-server)
in pants.engine.internals.graph.transitive_targets
in pants.engine.internals.graph.transitive_dependency_mapping
in pants.engine.internals.graph.resolve_targets (src/proto/helloworld.proto)
in pants.engine.internals.graph.resolve_unexpanded_targets (src/proto/helloworld.proto)
in pants.engine.internals.graph.resolve_dependencies (src/proto/helloworld.proto)
in pants.backend.codegen.protobuf.python.python_protobuf_subsystem.inject_dependencies (src/proto/helloworld.proto)
Traceback (most recent call last):
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 705, in native_engine_generator_send
res = func.send(arg)
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/backend/codegen/protobuf/python/python_protobuf_subsystem.py", line 129, in inject_dependencies
disable_inference_option=f"[{python_protobuf.options_scope}].infer_runtime_dependency",
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/backend/codegen/utils.py", line 56, in find_python_runtime_library_or_raise_error
else ""
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/util/strutil.py", line 240, in softwrap
if text[0] == "\n":
IndexError: string index out of range
(pants) [root@9a5752857756 pants-training]#
hundreds-father-404
08/19/2022, 5:24 PMpython_requirement
targets for protobuf
and possibly grpcio
. This step: https://www.pantsbuild.org/docs/protobuf-python#step-2-set-up-the-protobuf-and-grpcio-runtime-libraries
The bug is only preventing you from getting a good error message, but Pants would still be erroring either wayancient-rose-27306
08/19/2022, 5:43 PMin the meantime, this bug is triggered when you’re missingtargets forpython_requirement
and possiblyprotobuf
. This step: https://www.pantsbuild.org/docs/protobuf-python#step-2-set-up-the-protobuf-and-grpcio-runtime-librariesgrpcio
The bug is only preventing you from getting a good error message, but Pants would still be erroring either wayRegarding this, is it mandatory to have grpcio requiements within the same directory (or source root) as proto files? If you looked at my repo, I have the requirements.txt in
src/server
.hundreds-father-404
08/19/2022, 5:59 PMprotobuf
thoughancient-rose-27306
08/19/2022, 5:59 PMhundreds-father-404
08/19/2022, 6:00 PMprotobuf>=3.12.1
to requirements.txt
(the error message would have made this clear, if it triggered)ancient-rose-27306
08/19/2022, 6:00 PMhundreds-father-404
08/19/2022, 6:01 PMancient-rose-27306
08/19/2022, 6:01 PMhundreds-father-404
08/19/2022, 6:02 PMancient-rose-27306
08/19/2022, 6:03 PMAh that’s a good point; Pants’s eager check right now is too naive to recognize that, so it requires the explicit protobuf entry. But would you want to open a feature request to clean that up? https://github.com/pantsbuild/pants/issues/new/chooseCool, I will do that but I suppose it will take time for the feature to be released and so, I anyways have to explicitly mention protobuf.
hundreds-father-404
08/19/2022, 6:05 PMprotobuf
to your requirements.txt
After the PR is completeSorry, which PR? To improve pantsbuild/pants, or to add protobuf to your requirements.txt?
ancient-rose-27306
08/19/2022, 6:05 PMhundreds-father-404
08/19/2022, 6:07 PMprotobuf
to your requirements.txt
--
Generally, whenever a PR merges, you can set PANTS_SHA
to consume that commit. Or wait for a new Pants release https://www.pantsbuild.org/docs/manual-installation#running-pants-from-unreleased-buildsancient-rose-27306
08/19/2022, 6:45 PM./pants tailor
multiple times and the BUILDs will be updated automatically (including deletion of BUILD wherever not required anymore)?hundreds-father-404
08/19/2022, 6:48 PM./pants tailor
is solely additive: it will not remove or modify existing targetsancient-rose-27306
08/19/2022, 6:50 PMinstructions
though?hundreds-father-404
08/19/2022, 6:52 PMtailor
should automatically add docker_image
targets based on finding Dockerfile
. I'm not sure you need to mess with source roots for that to happen though, I'm pretty sure Docker doesn't really use source roots. That's more for JVM and Python
For instructions
, there is nothing that tailor
can autodetect. instructions
is solely meant as a way for you to define a Dockerfile
in your BUILD file, rather than a concrete Dockerfile
on diskancient-rose-27306
08/19/2022, 6:56 PMinstructions
, I am thinking from CI perspective. So, does that mean we need not execute ./pants tailor
in CI? I think I was mistaken in thinking that tailor needs to be executed everytime we run the build pipeline.hundreds-father-404
08/19/2022, 6:58 PMtailor --check
in CI because it can help make sure your developers aren't forgetting to add Pants targets
Whether you use instructions
or not does not really impact whether you should use tailor --check
in CI. There are other target types like python_sources
that tailor
addsancient-rose-27306
08/19/2022, 7:08 PMinstructions
automatically in CI. And so tailor need not (or rather should not) be run in CI. Thanks for informing about tailor --check
though. Same goes for other target types e.g. python_distribution()
or protobuf_sources()
for that matter where we have to write setup and grpc related things manually.hundreds-father-404
08/19/2022, 7:09 PMthere’s no way to add instructions automatically in CI.Taking a step back, we don't recommend that CI ever automatically adds targets. Your BUILD files should be created locally and checked in to version control That is, you should not run
./pants tailor
in CI, only ./pants tailor --check
. Does that make sense?ancient-rose-27306
08/19/2022, 7:09 PM./pants tailor
after moving requirements.txt
from src/server
to 3rdparty
(a new directory) and it gave following error:
(pants) [root@9a5752857756 pants-training]# ./pants tailor
19:10:09.99 [INFO] Initialization options changed: reinitializing scheduler...
19:10:10.87 [INFO] Scheduler initialized.
19:10:10.99 [ERROR] 1 Exception encountered:
Exception: Unmatched glob from python_requirements(address="src/server:reqs", description=None, module_mapping=FrozenDict({}), overrides=None, source=requirements.txt, tags=None, type_stubs_module_mapping=FrozenDict({}))'s field `source`: "src/server/requirements.txt"
Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.12/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
(pants) [root@9a5752857756 pants-training]#
Do you think its a good idea to always delete BUILD files and then re-tailor them?hundreds-father-404
08/19/2022, 7:14 PMpython_requirements()
target generator line from src/server/BUILD
to 3rdparty/BUILD
, since the code it describes requirements.txt
has now movedancient-rose-27306
08/19/2022, 7:24 PMhundreds-father-404
08/19/2022, 7:24 PMancient-rose-27306
08/19/2022, 7:25 PMpython_distribution()
to pex_binary()
for a certain target? Just add pex_binary()
to the existing BUILD and not just ever package
the previous python_distribution()
target?hundreds-father-404
08/19/2022, 7:26 PM./pants tailor
can't figure out what your intent isancient-rose-27306
08/19/2022, 7:27 PMSo, for safety, we never modify existing BUILD files. We only add to them
hundreds-father-404
08/19/2022, 7:29 PMJust add pex_binary() to the existing BUILD and not just ever package the previous python_distribution() target?Add the
pex_binary()
target, and then delete the python_distribution()
if you don't want to package that way anymoreancient-rose-27306
08/19/2022, 7:30 PMhundreds-father-404
08/19/2022, 7:32 PM./pants tailor
won't do it for youancient-rose-27306
08/19/2022, 7:34 PMprotobuf>=3.12.1
and simply protobuf
wouldn’t work? The current version of protobuf is 3.20package
failed even after mentioning this dependency in requirements.txt
.hundreds-father-404
08/19/2022, 9:10 PMprotobuf
matters?ancient-rose-27306
08/19/2022, 9:11 PMPANTS_SHA
before running the package
goal again and received following error now:
(pants) [root@9a5752857756 pants-training]# export PANTS_SHA=e43d8f655816351d11850b96d972034b2443b496
(pants) [root@9a5752857756 pants-training]# ./pants package src/server:grpc-server
Bootstrapping Pants using /root/miniconda3/envs/pants/bin/python3.7
Installing pantsbuild.pants==2.12.1rc3+gite43d8f65 into a virtual environment at /root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.1rc3+gite43d8f65_py37
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.12.1rc3+gite43d8f65 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0, 1.29.0, 1.30.0, 1.30.1, 1.30.2, 1.30.3, 1.30.4, 1.30.5rc1, 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1rc5, 2.5.1rc6, 2.5.1, 2.5.2rc0, 2.5.2rc1, 2.5.2rc2, 2.5.2rc3, 2.5.2, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0rc3, 2.6.0rc4, 2.6.0, 2.6.1rc0, 2.6.1rc1, 2.6.1rc2, 2.6.1rc3, 2.6.1, 2.7.0rc0, 2.7.0rc1, 2.7.0rc2, 2.7.0rc3, 2.7.0rc4, 2.7.0rc5, 2.7.0, 2.7.1rc0, 2.7.1rc1, 2.7.1, 2.7.2rc0, 2.7.2rc1, 2.7.2rc2, 2.7.2, 2.8.0.dev5, 2.8.0rc0, 2.8.0rc1, 2.8.0rc2, 2.8.0rc3, 2.8.0rc4, 2.8.0rc5, 2.8.0rc6, 2.8.0, 2.8.1rc0, 2.8.1rc1, 2.8.1rc2, 2.8.1, 2.9.0.dev0, 2.9.0.dev1, 2.9.0.dev2, 2.9.0.dev3, 2.9.0.dev4, 2.9.0rc0, 2.9.0rc1, 2.9.0rc2, 2.9.0rc3, 2.9.0rc4, 2.9.0rc5, 2.9.0rc6, 2.9.0, 2.9.1rc0, 2.9.1rc1, 2.9.1rc2, 2.9.1, 2.9.2rc0, 2.9.2, 2.10.0.dev0, 2.10.0.dev1, 2.10.0.dev2, 2.10.0.dev3, 2.10.0rc0, 2.10.0rc1, 2.10.0rc2, 2.10.0rc3, 2.10.0rc4, 2.10.0rc5, 2.10.0, 2.10.1rc0, 2.10.1rc1, 2.10.1, 2.11.0.dev0, 2.11.0.dev1, 2.11.0.dev2, 2.11.0.dev3, 2.11.0rc0, 2.11.0rc1, 2.11.0rc2, 2.11.0rc3, 2.11.0rc4, 2.11.0rc5, 2.11.0rc6, 2.11.0, 2.11.1rc0, 2.11.1rc1, 2.11.1rc2, 2.11.1rc3, 2.11.1, 2.12.0.dev0, 2.12.0.dev1, 2.12.0.dev2, 2.12.0.dev3, 2.12.0a0, 2.12.0rc0, 2.12.0rc1, 2.12.0rc2, 2.12.0rc3, 2.12.0, 2.12.1rc0, 2.12.1rc1, 2.12.1rc2, 2.12.1rc3, 2.13.0.dev0, 2.13.0.dev1, 2.13.0.dev2, 2.13.0.dev3, 2.13.0.dev4, 2.13.0.dev5, 2.13.0a0, 2.13.0a1, 2.13.0rc0, 2.13.0rc1, 2.14.0.dev0, 2.14.0.dev1, 2.14.0.dev2, 2.14.0.dev3, 2.14.0.dev4, 2.14.0.dev5, 2.14.0.dev6)
ERROR: No matching distribution found for pantsbuild.pants==2.12.1rc3+gite43d8f65
I set the env variable so that I am able to get a meaningful error message to be able to proceed with the package
. I am using RHEL distribution inside a docker container and 2.12.0 was working fine before setting the `PANTS_SHA`:
(pants) [root@9a5752857756 pants-training]# cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.6 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.6"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="<https://www.redhat.com/>"
DOCUMENTATION_URL="<https://access.redhat.com/documentation/red_hat_enterprise_linux/8/>"
BUG_REPORT_URL="<https://bugzilla.redhat.com/>"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
(pants) [root@9a5752857756 pants-training]#
Could you guys please help me here? My latest code is pushed in the repo.package
with `-ldebug`:
14:24:25.22 [DEBUG] Completed: Find interpreter for constraints: CPython==3.7.*
14:24:25.22 [DEBUG] Completed: Scheduling: Find interpreter for constraints: CPython==3.7.*
14:24:25.22 [DEBUG] Completed: Find Python interpreter for constraints - Selected /root/miniconda3/envs/pants/bin/python3.7 to run PEXes with.
14:24:25.22 [DEBUG] Completed: Scheduling: Determine Python dependencies for src/server/greeter_server.py
14:24:25.24 [DEBUG] Completed: Generate `python_requirement` targets from requirements.txt
14:24:25.25 [DEBUG] Completed: Find all targets in the project
14:24:25.25 [DEBUG] Completed: Find all Python targets in project
14:24:25.25 [DEBUG] Completed: Creating map of third party targets to Python modules
14:24:25.25 [DEBUG] Completed: Find all Protobuf targets in project
14:24:25.25 [DEBUG] Completed: Creating map of first party Python targets to Python modules
14:24:25.25 [DEBUG] Completed: Creating map of Protobuf targets to generated Python modules
14:24:25.25 [DEBUG] Completed: pants.backend.python.dependency_inference.module_mapper.merge_first_party_module_mappings
14:24:25.26 [DEBUG] Completed: Resolve transitive targets
14:24:25.26 [DEBUG] Completed: `package` goal
14:24:25.26 [DEBUG] computed 1 nodes in 1.616815 seconds. there are 219 total nodes.
14:24:25.26 [ERROR] 1 Exception encountered:
IndexError: string index out of range
(pants) [root@9a5752857756 pants-training]#
happy-kitchen-89482
08/22/2022, 7:13 PMmain
ancient-rose-27306
08/22/2022, 7:18 PMpackage
goal but could not find any relevant info.happy-kitchen-89482
08/22/2022, 7:21 PM./pants package src/client/::
successfully in your repo_pb2.py
files are in the resulting pexancient-rose-27306
08/22/2022, 7:24 PM./pants package src/server:grpc-server
./pants package src/client/::
(pants) [root@9a5752857756 pants-training]# ./pants package src/client/::
19:25:56.77 [ERROR] 1 Exception encountered:
IndexError: string index out of range
(pants) [root@9a5752857756 pants-training]#
$HOME/.cache/pants/
would help? That would mean I will be bootstrapping pants again.hundreds-father-404
08/22/2022, 8:41 PM./pants list ::
please?
@happy-kitchen-89482, indeed, the fix is only for the error message to be accurate rather than IndexError
. The underlying error is that protobuf
is not discoverable by Pants. The IndexError
was fixed in an unrelated PR in 2.13 and 2.14 alreadyhappy-kitchen-89482
08/22/2022, 8:52 PMancient-rose-27306
08/22/2022, 8:55 PMpackage
successfully.(pants) [root@9a5752857756 pants-training]# ./pants list ::
20:56:37.47 [INFO] Initializing scheduler...
20:56:37.79 [INFO] Scheduler initialized.
3rdparty:reqs
3rdparty:reqs#grpcio
3rdparty:reqs#grpcio-tools
3rdparty:reqs#protobuf
3rdparty/requirements.txt:reqs
src/client:client
src/client:greeter_client
src/client/greeter_client.py
src/proto:protos
src/proto/helloworld.proto:protos
src/server:server
src/server:docker
src/server:greeter_server
src/server:grpc-server
src/server/greeter_server.py
(pants) [root@9a5752857756 pants-training]#
hundreds-father-404
08/22/2022, 8:58 PMprotobuf
? Hm. Can you please run one of the commands that fails with --print-stacktrace
? (Or set print_stacktrace = true
in [GLOBAL]
in pants.toml
ancient-rose-27306
08/22/2022, 8:59 PM(pants) [root@9a5752857756 pants-training]# ./pants --print-stacktrace package src/server:grpc-server
20:59:01.84 [INFO] Initialization options changed: reinitializing scheduler...
20:59:02.48 [INFO] Scheduler initialized.
20:59:05.24 [ERROR] 1 Exception encountered:
Engine traceback:
in select
in pants.core.goals.package.package_asset
in pants.backend.python.goals.setup_py.package_python_dist (src/server:grpc-server)
in pants.engine.internals.graph.transitive_targets
in pants.engine.internals.graph.transitive_dependency_mapping
in pants.engine.internals.graph.resolve_targets (src/proto/helloworld.proto:protos)
in pants.engine.internals.graph.resolve_unexpanded_targets (src/proto/helloworld.proto:protos)
in pants.engine.internals.graph.resolve_dependencies (src/proto/helloworld.proto:protos)
in pants.backend.codegen.protobuf.python.python_protobuf_subsystem.inject_dependencies (src/proto/helloworld.proto:protos)
Traceback (most recent call last):
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 705, in native_engine_generator_send
res = func.send(arg)
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/backend/codegen/protobuf/python/python_protobuf_subsystem.py", line 144, in inject_dependencies
disable_inference_option=f"[{python_protobuf.options_scope}].infer_runtime_dependency",
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/backend/codegen/utils.py", line 56, in find_python_runtime_library_or_raise_error
else ""
File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.12.0_py37/lib/python3.7/site-packages/pants/util/strutil.py", line 240, in softwrap
if text[0] == "\n":
IndexError: string index out of range
(pants) [root@9a5752857756 pants-training]#
hundreds-father-404
08/22/2022, 9:04 PMpython_distribution
building via ./pants package src:grpc-server
.
It looks like there's a bug that for some reason we have to explicitly tell Pants that grpcio
provides grpc
, which it should recognize by default. I'm going to investigate this bug now
Then I also moved where the python_distribution
is defined up one directory. Because Pants supports multiple `python_distribution`s in the same repo, there is an algorithm to ensure that there are not multiple ambiguous owners of the same file. So, the error message prompted me to move the target up a BUILD level to avoid that.
In the process, Pants wanted me to tweak your source roots. Now, you would import server.greeter_server
, whereas before it was only greeter_server
. I suspect this is what you want. I kept it that Protobuf is helloworld_pb2
rather than proto.helloworld_pb2
thoughancient-rose-27306
08/22/2022, 9:45 PMhundreds-father-404
08/22/2022, 9:48 PMsrc/{proto,client,server}/BUILD
, which are giving metadata for the individual .py
and .proto
files via the protobuf_sources
and python_sources
targets
You also have src/BUILD
, which is where you declare to Pants that you want to build a python_distribution
made up of code inside the folder src/server
, which itself depends on code from src/proto
. Does that make sense?ancient-rose-27306
08/22/2022, 9:49 PMhundreds-father-404
08/22/2022, 9:50 PMancient-rose-27306
08/22/2022, 9:50 PMhundreds-father-404
08/22/2022, 9:50 PMpython_distribution
target
Then I also moved where the python_distribution is defined up one directory. Because Pants supports multiple python_distributions in the same repo, there is an algorithm to ensure that there are not multiple ambiguous owners of the same file. So, the error message prompted me to move the target up a BUILD level to avoid that.
ancient-rose-27306
08/22/2022, 9:52 PMNoOwnerError: No python_distribution target found to own src/proto/helloworld.proto:protos. Note that the owner must be in or above the owned target's directory, and must depend on it (directly or indirectly). See <https://www.pantsbuild.org/v2.12/docs/python-distributions> for how python_sources targets are mapped to distributions. See <https://www.pantsbuild.org/v2.12/docs/python-distributions>.
I got this after adding in grpc
to the requirements.hundreds-father-404
08/22/2022, 9:52 PMancient-rose-27306
08/22/2022, 9:54 PMprotobuf
. I got that.
Can this NoOwnerError
be resolved by defining a python_distribution()
for protos separately and not defining a BUILD at src?hundreds-father-404
08/22/2022, 9:55 PMOk, so I was getting the index error because there was one more requirement missing even after I added protobuf . I got that.Which is a bug. You already had
grpcio
in your requirements.txt
, and Pants by default is supposed to know that grpcio
provides the module grpc
. I'm trying to figure out now why this is not workingancient-rose-27306
08/22/2022, 9:55 PMhundreds-father-404
08/22/2022, 9:56 PMCan this NoOwnerError be resolved by defining a python_distribution() for protos separately and not defining a BUILD at src?Yep, that is 100% valid too! Then, Pants will detect that `src/server`'s distribution depends on the
src/proto
distribution.ancient-rose-27306
08/22/2022, 9:57 PMhundreds-father-404
08/22/2022, 9:58 PMsrc/client
depends on src/proto
also, Pants will detect that it too depends on the distribution in src/proto
ancient-rose-27306
08/22/2022, 9:58 PMhundreds-father-404
08/22/2022, 9:59 PMancient-rose-27306
08/22/2022, 10:00 PMExactly, and I think your idea is better than mine. That way, ifThe client indeed depends on proto and that’s why I was concerned about a BUILD in src. However, I think that can be overcome by adding in anotherdepends onsrc/client
also, Pants will detect that it too depends on the distribution insrc/proto
src/proto
python_distribution()
in src/BUILD
for client and we should be fine…no?hundreds-father-404
08/22/2022, 10:01 PMsrc/BUILD
and defining a python_distribution
in src/proto
. Also revert my source root changes maybe
It's not legal for two `python_distribution`s to "own" the same code, in this case src/proto/*
ancient-rose-27306
08/22/2022, 10:05 PMIt’s not legal for two `python_distribution`s to “own” the same code, in this caseDoes that mean I can’t have two `python_distribution()`s insrc/proto/*
src/BUILD
(one each for client and server)?
This concept of ownership is new to me as I am learning it for the first time now.hundreds-father-404
08/22/2022, 10:09 PMpython_distribution
targets in src/BUILD
-- the important thing is that because src/proto
is used by both distributions, that you have the dedicated python_distribution
for it. That way, src/client
and src/server
can unambiguously depend on the same thing. Does that make sense?
(Btw, this is a fairly wonky part of Pants that is only relevant to the python_distribution
target, due to publishing semantics)That way, src/client and src/server can unambiguously depend on the same thing.Alternatively, I think you can have either
src/client
or src/server
be the "owner" of the Protobuf file; then the other will have to depend on that one, like src/client
must depend on src/server
. Which I imagine isn't what you wantancient-rose-27306
08/22/2022, 10:11 PMsrc/BUILD
like below:
python_distribution(
name="grpc-server",
dependencies=["src/server:server"],
sdist=False,
provides=python_artifact(
name="grpc-server",
version="0.1.0",
)
)
python_distribution(
name="grpc-client",
dependencies=["src/client:client"],
sdist=False,
provides=python_artifact(
name="grpc-client",
version="0.1.0",
)
)
python_distribution(
name="proto-lib",
dependencies=["src/proto:protos"],
sdist=False,
provides=python_artifact(
name="proto-lib",
version="0.1.0",
)
)
Where’s the ambiguity coming from? Each target has their own name. I am not going to use it like this as I prefer separate BUILD files but just for the purpose of understanding. And this change would have me modify the source_roots as only src/
, I believe I will have to change the import statements to:
import proto.helloworld_pb2
import proto.helloworld_pb2_grpc
hundreds-father-404
08/22/2022, 10:18 PM./pants package src:
? (In 2.13, you can simply do ./pants package src
to say "build everything in this directory")ancient-rose-27306
08/22/2022, 10:18 PMhundreds-father-404
08/22/2022, 10:19 PMI believe I will have to change the import statements to:Or, set source roots to
/src
and /src/proto
. Pants uses the most precise source root possible for the file in question. Although that might cause more confusion than it's worthNo, I haven’t tried it yet. I was just trying to understand it.I believe what you have above should work
ancient-rose-27306
08/22/2022, 10:22 PM"/src
and modified the import statements. I have three libraries now as opposed to just single wheel before for the server because earlier, I only built server and now, I built src. I believe I could build just one wheel even now.hundreds-father-404
08/22/2022, 10:23 PMancient-rose-27306
08/22/2022, 10:23 PM(pants) [root@9a5752857756 pants-training]# ./pants package src:grpc-client
22:22:58.82 [INFO] Wrote dist/grpc_client-0.1.0-py3-none-any.whl
hundreds-father-404
08/22/2022, 10:24 PMIndexError
and everything, and I'm glad we can fix this for future folks trying it outancient-rose-27306
08/22/2022, 10:25 PMname="grpc-client"
in my python_distribution()
but the whl is getting created with the name grpc_client (-
replaced by _
). This is probably to adhere to python standards.I’m really glad you reached out for help! That definitely was not a very good first impression with theI can’t thank you and @happy-kitchen-89482 enough to help me move through this. I am asking very basic questions and you guys are so patient to answer each of them.and everything, and I’m glad we can fix this for future folks trying it outIndexError
hundreds-father-404
08/22/2022, 10:28 PMgrpcio
in the default module mapping until Pants 2.14.0.dev5 https://github.com/pantsbuild/pants/pull/16337. I will cherry-pick this fix to 2.12 and 2.13. Thank you again so much for the report!ancient-rose-27306
08/22/2022, 10:41 PMhundreds-father-404
08/22/2022, 10:41 PMsalmon-planet-80683
09/02/2022, 2:48 PMhundreds-father-404
09/02/2022, 3:23 PMgrpc
in the defaults, so you can now remove the module_mapping
if you want 🙂ancient-rose-27306
09/02/2022, 3:31 PM