https://pantsbuild.org/ logo
Join Slack
Powered by
Have a strange issue. I have a pants build that in...
# general
p
Have a strange issue. I have a pants build that includes TensorFlow 1.15 which transitively depends on gast==0.2.2. That exists on pypi for Python 3.7 and I can enter a py3.7 venv and 
pip install -r constraints.txt
without issues. However, when I try to 
pants package
I get:
Copy code
stderr:
ERROR: Could not find a version that satisfies the requirement gast==0.2.2
ERROR: No matching distribution found for gast==0.2.2
pid 794090 -> /home/oliver/.cache/pants/named_caches/pex_root/venvs/7f48ea14fe9a31ad7cc2422df7a949fe8899372d/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root --log /tmp/process-executionMrBaUB/.tmp/tmpmhp64qj_/pip.log download --dest /tmp/process-executionMrBaUB/.tmp/tmpq73a30sv/macosx_10_14_x86_64-cp-37-cp37m --platform macosx_10_14_x86_64 --implementation cp --python-version 37 --abi cp37m --only-binary :all: --constraint constraints.txt absl-py==0.13.0 astor==0.8.1 attrs==21.2.0 backoff==1.10.0 cachetools==4.2.2 certifi==2021.5.30 chardet==4.0.0 gast==0.2.2 google-api-core==1.30.0 google-auth==1.31.0 google-cloud-core==1.7.0 google-cloud-firestore==2.1.3 google-pasta==0.2.0 googleapis-common-protos==1.53.0 grpcio==1.38.0 h5py==2.10.0 idna==2.10 importlib-metadata==4.5.0 iniconfig==1.1.1 keras-applications==1.0.8 keras-preprocessing==1.1.2 labelbox==2.6.0 markdown==3.3.4 ndjson==0.3.1 numpy==1.18.5 opencv-python-headless==4.5.2.54 opt-einsum==3.3.0 packaging==20.9 pluggy==0.13.1 proto-plus==1.18.1 protobuf==3.17.3 py==1.10.0 pyasn1-modules==0.2.8 pyasn1==0.4.8 pydantic==1.8.2 pyparsing==2.4.7 pytest==6.2.4 pytz==2021.1 requests==2.25.1 rsa==4.7.2 six==1.16.0 tensorboard==1.15.0 tensorflow-estimator==1.15.1 tensorflow==1.15.5 termcolor==1.1.0 toml==0.10.2 typing-extensions==3.10.0.0 urllib3==1.26.5 werkzeug==2.0.1 wheel==0.36.2 wrapt==1.12.1 zipp==3.4.1 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None
I re-ran with 
--no-process-execution-local-cleanup
but the 
pip.log
file it's supposed to create in 
.tmp
doesn't exist.
w
what interpreter constraints are you using in this repo and on these targets?
p
Copy code
[python-setup]
interpreter_constraints = ['CPython==3.7.*']
requirement_constraints = 'constraints.txt'
We're forced to stick with 3.7 due to dependency on an old version of TensorFlow.
w
got it. and is there any more to the error message?
p
Here's the full output:
Copy code
(venv) oliver@marmaduke:~/Documents/code/central2/python$ ./pants package db_cli/::
10:51:26.74 [INFO] Completed: Resolving constraints.txt
10:51:26.74 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 498, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Resolving constraints.txt' failed with exit code 1.
stdout:

stderr:
ERROR: Could not find a version that satisfies the requirement gast==0.2.2
ERROR: No matching distribution found for gast==0.2.2
pid 795584 -> /home/oliver/.cache/pants/named_caches/pex_root/venvs/7f48ea14fe9a31ad7cc2422df7a949fe8899372d/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root --log /tmp/process-executionlRKkVb/.tmp/tmpyfr58o_p/pip.log download --dest /tmp/process-executionlRKkVb/.tmp/tmpjmhkeakd/macosx_10_14_x86_64-cp-37-cp37m --platform macosx_10_14_x86_64 --implementation cp --python-version 37 --abi cp37m --only-binary :all: --constraint constraints.txt absl-py==0.13.0 astor==0.8.1 attrs==21.2.0 backoff==1.10.0 cachetools==4.2.2 certifi==2021.5.30 chardet==4.0.0 gast==0.2.2 google-api-core==1.30.0 google-auth==1.31.0 google-cloud-core==1.7.0 google-cloud-firestore==2.1.3 google-pasta==0.2.0 googleapis-common-protos==1.53.0 grpcio==1.38.0 h5py==2.10.0 idna==2.10 importlib-metadata==4.5.0 iniconfig==1.1.1 keras-applications==1.0.8 keras-preprocessing==1.1.2 labelbox==2.6.0 markdown==3.3.4 ndjson==0.3.1 numpy==1.18.5 opencv-python-headless==4.5.2.54 opt-einsum==3.3.0 packaging==20.9 pluggy==0.13.1 proto-plus==1.18.1 protobuf==3.17.3 py==1.10.0 pyasn1-modules==0.2.8 pyasn1==0.4.8 pydantic==1.8.2 pyparsing==2.4.7 pytest==6.2.4 pytz==2021.1 requests==2.25.1 rsa==4.7.2 six==1.16.0 tensorboard==1.15.0 tensorflow-estimator==1.15.1 tensorflow==1.15.5 termcolor==1.1.0 toml==0.10.2 typing-extensions==3.10.0.0 urllib3==1.26.5 werkzeug==2.0.1 wheel==0.36.2 wrapt==1.12.1 zipp==3.4.1 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None



(Use --print-stacktrace to see more error details.)
I did just figure out that this seems to fail with only one of my targets; specifically one that's trying to build a cross-platform .pex for Linux and OSX. This built fine with pants 2.4 - I just upgraded to 2.5 and now it fails. That may, or may not, be a red herring.
w
…ah. yea, that is quite relevant.
p
Note that the one build that fails has no dependencies on TensorFlow or gast. But I think maybe it tries to resolve all the 
constraints.txt
even if the target being built doesn't need it??
Yeah - the only target that fails is the one cross-platform pex
w
But I think maybe it tries to resolve all the 
constraints.txt
 even if the target being built doesn’t need it?
iirc, only if you set a non-default option… sec
p
Copy code
platforms = [
      'linux-x86_64-cp-37-m',
      'macosx-10.14-x86_64-cp-37-m'
   ]
Again, note that this .pex doesn't actually have any dependencies on gast and, even if it did, gast appears to be pure-python so there's no platform-specific dependency.
w
it should not resolve the entire constraints file at once unless you set https://www.pantsbuild.org/docs/reference-python-setup#section-resolve-all-constraints … which is a fairly experimental option
p
I did not set that option
I did just verify that the one target that's failing will build if I remove 
macosx
from the 
platforms
.
Going to try adding it back to platforms but build with pants 2.4
w
OH. i lied. apparently 
resolve_all_constraints
does default to enabled.
p
interesting.
w
so, it sounds like there might be a bug related to attempting resolve the entire constraints file for all relevant platforms.
p
can now confirm that the target in question does build, even with the 
macosx
platform on Pants 2.4
and that bug seems to be 2.5 specific.
Looked at your profile; looks like you're one of the pants devs (thanks for your work!!). Do you want me to file a bug or is it easier if you do?
❤️ 1
w
@plain-carpet-73994: i’m just back from leave, so missing some context on recent changes. would you mind putting a tiny sketch of a repro in a ticket? i think that i see a workaround, but want to actually track it
p
sure. Link to your bug tracker?
w
seems likely to be related to the changes that lead to https://github.com/pantsbuild/pants/pull/11985, but i can link once it’s in
p
Thanks for your help @witty-crayon-22786!!
w
thanks for the report!
h
@plain-carpet-73994 try setting 
[python-setup].resolve_all_constraints = "nondeployables"
for now on 2.5 please. You'll get a deprecation warning, but I want to check it fixes the issue
w
@hundreds-father-404: I was going to suggest that, but it looked like John had effectively already removed the implementation?
h
not deleted till 2.6
👍 1
p
@hundreds-father-404 I looked at the bug and it sounds like the current thinking is now that this is a lock-file issue which prevents the dependencies for two different architectures from being resolved simultaneously. Given that, would the 
resolve_all_constraints
experiment still be valuable?
w
yes: the effect of setting the value eric mentioned above: https://pantsbuild.slack.com/archives/C046T6T9U/p1623955403419200?thread_ts=1623951753.412200&amp;cid=C046T6T9U … is that deploy binaries would get their own resolves
5 Views
Open in Slack
PreviousNext
Powered by