https://pantsbuild.org/ logo
Join Slack
Powered by
Got some build/bootstrap issue with pants now (doi...
# development
c
Got some build/bootstrap issue with pants now (doing pants development) on latest main. First I had to 
brew install osxfuse
but that was an easy one. Now, it fails with:
Copy code
Building wheels for collected packages: setproctitle
pex.jobs.Error: Executing /usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -s -E ...
[... see thread for full log ...]
I found this a bit further up in the log, not sure if it is related:
Copy code
Building wheels for collected packages: setproctitle
[... see thread for full log ...]
  ValueError: invalid literal for int() with base 10: "['11']"
not sure what to make of it..
if relevant, these are my python versions:
Copy code
$ python --version
Python 2.7.17
$ python3 --version
Python 3.9.2
$ python3.8 --version
Python 3.8.5
Moved lengthy logs from main post into thread…
Copy code
Created temporary directory: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-unpack-wrmbcqp7
Building wheels for collected packages: setproctitle
  Building wheel for setproctitle (PEP 517): finished with status 'done'
  Created temporary directory: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-wheel-incspmpy
  Destination directory: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-wheel-incspmpy
  Building wheel for setproctitle (PEP 517): started
  Created wheel for setproctitle: filename=setproctitle-1.2.2-cp38-cp38-macosx_10_15_x86_64.whl size=10969 sha256=dbc6094da9abbeb3c3d9c1758ace2760d5c1d6b62e3212d65732deffbcc520fa
  Running command /Applications/Xcode.app/Contents/Developer/usr/bin/python3 /Users/x/.cache/pants/named_caches/pex_root/pip.pex/7e33e3c358a36820c17df357611df4c7a8e7d66e/.deps/pip/pip/_vendor/pep517/_in_process.py build_wheel /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/tmp21cwsmty
  Stored in directory: /Users/x/.cache/pants/named_caches/pex_root/wheels/58/5e/16/6ff76e543b02443b174381586fe850dfc4034c11f9018c8bd6
Successfully built setproctitle
Removed build tracker: '/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-req-tracker-it0z5pyw'
Traceback (most recent call last):
  File "/Users/x/.cache/pants/named_caches/pex_root/unzipped_pexes/406ff2dc55d7d4a06e180299256979a672296983/.deps/pex-2.1.38-py2.py3-none-any.whl/pex/jobs.py", line 431, in execute_parallel
    yield spawn_result.spawned_job.await_result()
  File "/Users/x/.cache/pants/named_caches/pex_root/unzipped_pexes/406ff2dc55d7d4a06e180299256979a672296983/.deps/pex-2.1.38-py2.py3-none-any.whl/pex/jobs.py", line 185, in await_result
    job.wait()
  File "/Users/x/.cache/pants/named_caches/pex_root/unzipped_pexes/406ff2dc55d7d4a06e180299256979a672296983/.deps/pex-2.1.38-py2.py3-none-any.whl/pex/jobs.py", line 74, in wait
    self._check_returncode()
  File "/Users/x/.cache/pants/named_caches/pex_root/unzipped_pexes/406ff2dc55d7d4a06e180299256979a672296983/.deps/pex-2.1.38-py2.py3-none-any.whl/pex/jobs.py", line 123, in _check_returncode
    raise self.Error(
pex.jobs.Error: Executing /usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -s -E /Users/x/.cache/pants/named_caches/pex_root/pip.pex/7e33e3c358a36820c17df357611df4c7a8e7d66e --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -v --cache-dir /Users/x/.cache/pants/named_caches/pex_root wheel --no-deps --wheel-dir /Users/x/.cache/pants/named_caches/pex_root/built_wheels/sdists/setproctitle-1.2.2.tar.gz/bdcd4b65f16fe6f53f58ee32cea0532d6839c103/cp39-cp39.0181788706594c4eb3edf605e1fb8299 /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/tmpsl2dh0nb/usr.local.Cellar.python@3.9.3.9.2_1.Frameworks.Python.framework.Versions.3.9.bin.python3.9/setproctitle-1.2.2.tar.gz --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 failed with 1
Copy code
Building wheels for collected packages: setproctitle
  Created temporary directory: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-wheel-0ufw65fo
  Destination directory: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-wheel-0ufw65fo
  Building wheel for setproctitle (PEP 517): started
  Running command /usr/local/opt/python@3.8/bin/python3.8 /Users/aadt/.cache/pants/named_caches/pex_root/pip.pex/7e33e3c358a36820c17df357611df4c7a8e7d66e/.deps/pip/pip/_vendor/pep517/_in_process.py build_wheel /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/tmpa1k7a4dy
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -DSPT_VERSION=1.2.2 -D__darwin__=1 -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/tcl-tk/include -I/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/spt_status.c -o build/temp.macosx-['11'].0-x86_64-3.9/src/spt_status.o
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -DSPT_VERSION=1.2.2 -D__darwin__=1 -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/tcl-tk/include -I/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/spt_strlcpy.c -o build/temp.macosx-['11'].0-x86_64-3.9/src/spt_strlcpy.o
  creating build/lib.macosx-['11'].0-x86_64-3.9
  clang -bundle -undefined dynamic_lookup build/temp.macosx-['11'].0-x86_64-3.9/src/setproctitle.o build/temp.macosx-['11'].0-x86_64-3.9/src/spt_debug.o build/temp.macosx-['11'].0-x86_64-3.9/src/spt_setup.o build/temp.macosx-['11'].0-x86_64-3.9/src/spt_status.o build/temp.macosx-['11'].0-x86_64-3.9/src/spt_strlcpy.o -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -L/usr/local/opt/tcl-tk/lib -o build/lib.macosx-['11'].0-x86_64-3.9/setproctitle.cpython-39-darwin.so
  installing to build/bdist.macosx-['11'].0-x86_64/wheel
  running install
  running install_lib
  creating build/bdist.macosx-['11'].0-x86_64
  creating build/bdist.macosx-['11'].0-x86_64/wheel
  copying build/lib.macosx-['11'].0-x86_64-3.9/setproctitle.cpython-39-darwin.so -> build/bdist.macosx-['11'].0-x86_64/wheel
  running install_egg_info
  running egg_info
  writing setproctitle.egg-info/PKG-INFO
  writing dependency_links to setproctitle.egg-info/dependency_links.txt
  writing requirements to setproctitle.egg-info/requires.txt
  writing top-level names to setproctitle.egg-info/top_level.txt
  reading manifest file 'setproctitle.egg-info/SOURCES.txt'
  reading manifest template '<http://MANIFEST.in|MANIFEST.in>'
  warning: no files found matching 'Makefile'
  writing manifest file 'setproctitle.egg-info/SOURCES.txt'
  Copying setproctitle.egg-info to build/bdist.macosx-['11'].0-x86_64/wheel/setproctitle-1.2.2-py3.9.egg-info
  running install_scripts
  Traceback (most recent call last):
    File "/Users/x/.cache/pants/named_caches/pex_root/pip.pex/7e33e3c358a36820c17df357611df4c7a8e7d66e/.deps/pip/pip/_vendor/pep517/_in_process.py", line 280, in <module>
  running bdist_wheel
      main()
    File "/Users/x/.cache/pants/named_caches/pex_root/pip.pex/7e33e3c358a36820c17df357611df4c7a8e7d66e/.deps/pip/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/x/.cache/pants/named_caches/pex_root/pip.pex/7e33e3c358a36820c17df357611df4c7a8e7d66e/.deps/pip/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 221, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 207, in _build_with_temp_dir
      self.run_setup()
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 258, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 84, in <module>
      setup(
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
  running build
  running build_ext
      self.run_command(cmd)
    File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 337, in run
      impl_tag, abi_tag, plat_tag = self.get_tag()
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 260, in get_tag
      plat_name = get_platform(self.bdist_dir)
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 52, in get_platform
      result = calculate_macosx_platform_tag(archive_root, result)
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/wheel/macosx_libfile.py", line 366, in calculate_macosx_platform_tag
      base_version = tuple([int(x) for x in base_version.split(".")])
    File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-execution6L2CoA/.tmp/pip-build-env-frfw62kz/overlay/lib/python3.9/site-packages/wheel/macosx_libfile.py", line 366, in <listcomp>
      base_version = tuple([int(x) for x in base_version.split(".")])
  ValueError: invalid literal for int() with base 10: "['11']"
  building 'setproctitle' extension
  creating build
  creating build/temp.macosx-10.15-x86_64-3.8
h
Oh, what macOS version are you using? I think this is because the latest version uses 11 instead of 10 for the OS major version You may need to upgrade pip. I think this is in the bash script virtualenv. You should see that we install wheel there. Try also upgrading pip in it
The main change from the past week that would impact this is we now use python -m venv to create the venv, rather than virtualenv. I suspect your venv's pip is outdated and doesn't know how to handle your macOS version
c
ah, you may be spot on there, as I did see a pip outdated log fly by
I’m on big sur 11.2.3
h
If that does fix it, a PR would be much appreciated to update that one place and to also update our release script, which uses the 
venv
module as a library - you'd add 
subprocess.run([..path/bin/pip, install, --upgrade, pip], check=True)
etc to the two functions here https://github.com/pantsbuild/pants/blob/f4c148b07ae7aa31b262e9245b7b8805b6de2fbd/build-support/bin/packages.py#L278-L296 (looks like line 296 is missing 
check=True
, oops)
c
didn’t seem to do any good (or I failed to apply your suggestion) also, it seems to still use py3.9 although I’ve exported 
PY=python3.8
h
How were these python interpreters installed?
c
using brew
except 2.7, which is the default that comes with the system, I think..
h
Oh, @curved-television-6568 what command are you running? I realized the log is a Pex log and it's talking about Pex's vendored version of pip, which explains why upgrading pip in 
virtualenv
+ changing 
$PY
didn't do anything
c
First it was just the pre commit hook, but I narrowed it down so I get the error running typecheck on a single file.
👍 1
Lint works fine.
h
Ah ha: Pip 20.3 adds support for macOS Big Sur compatibility tags: https://pip.pypa.io/en/stable/news/#id48v Pex is vendoring Pip 20.2.4 https://github.com/pantsbuild/pex/blob/main/tox.ini#L85 cc @enough-analyst-54434, does that sound reasonable for us to upgrade to 20.3.4? Ik we can't yet do 21.x.y because it drops Py2 support
Andreas, in the meantime, you can skip the pre commit checks with 
--no-verify
. It's still good to run 
./pants --changed-since=HEAD lint
though
👍 1
e
The tox is just to run the vendoring tool, no relation to what we vendor. That's here: https://github.com/pantsbuild/pex/blob/4f7bf307e493f8b45cbf490604c5b2aa4f457dfa/pex/vendor/__init__.py#L102-L126
IOW we're on the latest possible and last Pip to support Python 2.7 + 1 patch.
👍 1
I have not scanned above, but its clearly a different problem. We do vendor old setuptools. Is it that?
44.0.0 - also last to support Python 2.7, but they cut over much earlier than Pip did.
👍 1
These threads are ... not awesome. Is it time to file an issue with narrowed down relevant details?
c
Agree. I’m reluctant to open issues until I know it is an actual issue.. (the error usually lies closer to home, most of the time in my experience 😉 ) Filed an issue for this now, though: https://github.com/pantsbuild/pants/issues/11967 Hope I got the relevant details out, but just in case, I attached a log file from a run (some 18Mb of it!) so you shouldn’t have to wait for me to wake up for more details if needed. If there’s anything more you need, or want me to run some diagnostics etc, just let me know, happy to do what I can from my end.
Open in Slack
PreviousNext
Powered by