https://pantsbuild.org/ logo
Join Slack
Powered by
Getting the following when running `./pants` with...
# development
c
Getting the following when running 
./pants
with version set to 2.8.0.dev4 . 2.8.0.dev3 works fine.
Copy code
ERROR: Command errored out with exit status 1:
   command: /Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.2H81Or/install/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-install-tgl3794i/pystache_31f7891a3aad43f38c9c7f0b3c203d9d/setup.py'"'"'; __file__='"'"'/private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-install-tgl3794i/pystache_31f7891a3aad43f38c9c7f0b3c203d9d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-pip-egg-info-5bh87jeb
       cwd: /private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-install-tgl3794i/pystache_31f7891a3aad43f38c9c7f0b3c203d9d/
  Complete output (3 lines):
  pystache: using: version '58.0.4' of <module 'setuptools' from '/Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.2H81Or/install/lib/python3.9/site-packages/setuptools/__init__.py'>
  Warning: 'classifiers' should be a list, got type 'tuple'
  error in pystache setup command: use_2to3 is invalid.
w
so, i have seen this recently, and i think that it is because 
setuptools
can float in virtualenvs …
one sec.
hm: i don’t repro with 3.9.
@clean-city-64472: how old is your 
pants
script? this sounds a bit like something that might have been addressed there.
c
brand new today
w
oof. alrighty then.
c
i changed 
pants_version
to dev3 and rerun the exact same command and it all works
switched back to dev4 and i get the error again
w
the script creates virtualenvs in 
~/.cache/pants/setup
… if you move them aside, do you see the same error for both dev3 and dev4?
c
Copy code
# pants_version = 2.8.0.dev3
rm -rf ~/.cache/pants/setup
./pants 
# Works

# pants_version = 2.8.0.dev4
rm -rf ~/.cache/pants/setup
./pants
# Error
pystache
doesn't show up in the output for dev3
w
ok, can you include more of the output from the error run? there should be some more references to 
setuptools
c
Copy code
Bootstrapping Pants using /usr/local/bin/python3.9
Creating the virtualenv PEX.
Downloading the Pex PEX.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   609  100   609    0     0   1371      0 --:--:-- --:--:-- --:--:--  1371
100 3529k  100 3529k    0     0  3550k      0 --:--:-- --:--:-- --:--:-- 9794k
SHA256 fingerprint of <https://github.com/pantsbuild/pex/releases/download/v2.1.42/pex> verified.
Installing pantsbuild.pants==2.8.0.dev4 into a virtual environment at /Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/2.8.0.dev4_py39
created virtual environment CPython3.9.1.final.0-64 in 258ms
  creator CPython3Posix(dest=/Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.7Os7rG/install, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/gordon/Library/Application Support/virtualenv)
    added seed packages: pip==21.2.4, setuptools==58.0.4, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Requirement already satisfied: pip in /Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.7Os7rG/install/lib/python3.9/site-packages (21.2.4)
Collecting pip
  Using cached pip-21.3-py3-none-any.whl (1.7 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-21.3
Collecting pantsbuild.pants==2.8.0.dev4
  Using cached pantsbuild.pants-2.8.0.dev4-cp39-cp39-macosx_10_15_x86_64.whl (9.4 MB)
Collecting pex==2.1.51
  Using cached pex-2.1.51-py2.py3-none-any.whl (2.6 MB)
Collecting setproctitle==1.2.2
  Using cached setproctitle-1.2.2-cp39-cp39-macosx_10_15_x86_64.whl
Collecting ijson==3.1.4
  Using cached ijson-3.1.4-cp39-cp39-macosx_10_9_x86_64.whl (52 kB)
Collecting types-PyYAML==5.4.3
  Using cached types_PyYAML-5.4.3-py2.py3-none-any.whl (12 kB)
Collecting types-setuptools==57.0.0
  Using cached types_setuptools-57.0.0-py3-none-any.whl (4.8 kB)
Collecting psutil==5.8.0
  Using cached psutil-5.8.0-cp39-cp39-macosx_10_9_x86_64.whl (236 kB)
Collecting types-toml==0.1.3
  Using cached types_toml-0.1.3-py2.py3-none-any.whl (2.1 kB)
Collecting pystache==0.5.4
  Using cached pystache-0.5.4.tar.gz (75 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.7Os7rG/install/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-install-r83h41d7/pystache_afeb63400dc04fa6945ef47635299500/setup.py'"'"'; __file__='"'"'/private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-install-r83h41d7/pystache_afeb63400dc04fa6945ef47635299500/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-pip-egg-info-6stlpqok
       cwd: /private/var/folders/g4/8ps3_1d52wl2h7g4120w1yy40000gn/T/pip-install-r83h41d7/pystache_afeb63400dc04fa6945ef47635299500/
  Complete output (3 lines):
  pystache: using: version '58.0.4' of <module 'setuptools' from '/Users/gordon/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.7Os7rG/install/lib/python3.9/site-packages/setuptools/__init__.py'>
  Warning: 'classifiers' should be a list, got type 'tuple'
  error in pystache setup command: use_2to3 is invalid.
  ----------------------------------------
WARNING: Discarding <https://files.pythonhosted.org/packages/d6/fd/eb8c212053addd941cc90baac307c00ac246ac3fce7166b86434c6eae963/pystache-0.5.4.tar.gz#sha256=f7bbc265fb957b4d6c7c042b336563179444ab313fb93a719759111eabd3b85a> (from <https://pypi.org/simple/pystache/>). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pystache==0.5.4 (from pantsbuild-pants) (from versions: 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4)
ERROR: No matching distribution found for pystache==0.5.4
w
oh, you’re right: that is a very important difference.
c
Interesting - in dev3 it looks like it uninstalls setuptools 58 and reinstalls 57. That happens near the end - after other deps are installed.
w
i think that that combines with 
pystache
not being compatible with some 
setuptools
versions (after 
58
: and your machine has 
58.0.4
in the initial venv)
c
Ic - anyway dev3 gives me everything I need so I'm not blocked or anything
w
so… two things. 1) we should pin the setuptools we use inside the venv before installing deps (or switch to distributing a PEX, because sheesh), 2) we should stop depending on 
pystache
👍 2
f
unclear if relevant, we just started using pystache in dev4 for the Go plugin to produce the main function for test binaries.
h
Oh, yeah, I think that was meant as a build-support only type of dep @fast-nail-55400, I didn't realize that in code review
w
yea, it's ancient and unsupported unfortunately =(
h
As I think we know, Pants definitely needs a visibility feature to avoid issues like this
f
so we need to switch off of pystache?
w
f
I can switch us off of pystache easily enough. Any suggestions for template libraries? We could also just generate the main function in Go using Go’s template library (since we already run a Go wrapper to analyze the test sources).
👍 1
h
that's in its std lib? Probably that approach. Whenever you add a new dep to Pants, it slows down Pants install times, which hits users' CI
f
yes it is a part of Go’s stdlib
👍 1
Whenever you add a new dep to Pants, it slows down Pants install times, which hits users’ CI
not the case this time. I chose 
pystache
because it was already in the venv …
h
in our internal venv, not pantsbuild.pants dist's requirements. It's deceptive that not everything in our requirements.txt actually gets included in the dist
f
we should make those contexts clearer
➕ 1
or have a CI test that fails if you use an internal-only venv module or something
or a lint
in any event, I’ll switch the Go plugin to use Go’s template lib
h
Yes, agreed. Which is the visibility feature I'm talking about. We shouldn't just fix this for Pants itself, but give a new feature for all users This is a generally useful thing, make sure that a dev doesn't unintentionally add a dep the org doesn't want to add, like adding NumPy to something
w
...and I know what @clean-city-64472 is thinking: "if this had been in a 
BUILD
file we would have seen it!"... but yea: this is a case for some sort of visibility enforcement probably. because if you don't see it in imports, I doubt you'd see it in the BUILD file. and transitive still gets you.
➕ 1
c
I’m getting warmer on dependency inference 🙂
💯 2
🤘 2
h
I was thinking the same thing at first, but I don't think it would have saved us here. I knew we were adding a new dep to Go code, just didn't realize it wasn't in the dist already. Transitive deps were the culprit, not direct. explicit BUILD files only help with direct deps And indeed, that's a fundamental promise of Pants: you shouldn't have to reason about transitive deps! Let the tool do it for you
f
c
We got into a similar situation with the 
dockerfile-parser
.. just my +1 ;)
Open in Slack
PreviousNext
Powered by