https://pantsbuild.org/ logo
#general
Title
# general
b

brash-baker-91190

11/30/2021, 9:01 PM
Has anybody had experience running Pants on gitpod.io? I'm running into
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
when I try to run anything (even
./pants version
)
👀 1
h

hundreds-father-404

12/01/2021, 4:06 PM
Hey Chris, I suspect this might be when installing Pants via the bash script? Could you please add
set -x
to the top and try running again, copy the output if posible here?
h

hundreds-father-404

12/01/2021, 4:15 PM
Copy code
++ /home/gitpod/.cache/pants/setup/bootstrap-Linux-x86_64/pants.8SS4Vl/install/bin/pip install -U pip
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
WARNING: You are using pip version 21.1.2; however, version 21.3.1 is available.
You should consider upgrading via the '/home/gitpod/.cache/pants/setup/bootstrap-Linux-x86_64/pants.8SS4Vl/install/bin/python -m pip install --upgrade pip' command.
b

brash-baker-91190

12/01/2021, 4:15 PM
This is using Gitpod's base container image, which appears to use Pyenv for its Python environments. I'm able to run things under
sudo
and have it work, which ends up using a system Python (not Pyenv), if that's useful.
h

hundreds-father-404

12/01/2021, 4:18 PM
b

brash-baker-91190

12/01/2021, 4:25 PM
Initially got this error when running from the PEX:
Copy code
/usr/bin/env: 'python3.7': No such file or directory
Installing a 3.7 Python now....
After installing and enabling a 3.7 Python...
Copy code
$ pyenv versions
  system
* 3.7.0 (set by /workspace/cloudsmith-buildkite-plugin/.python-version)
  3.8.12
I get the following error:
Copy code
$ ./pants.2.8.1rc1.pex version
Traceback (most recent call last):
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 476, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 532, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 668, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 700, in execute_pkg_resources
    return runner()
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_loader.py", line 115, in main
    PantsLoader.main()
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_loader.py", line 111, in main
    cls.run_default_entrypoint()
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_loader.py", line 93, in run_default_entrypoint
    exit_code = runner.run(start_time)
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_runner.py", line 77, in run
    with initialize_stdio(global_bootstrap_options), stdio_destination(
  File "/home/gitpod/.pyenv/versions/3.7.0/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/init/logging.py", line 167, in initialize_stdio
    global_bootstrap_options.pants_workdir,
  File "/home/gitpod/.pyenv/versions/3.7.0/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/init/logging.py", line 204, in initialize_stdio_raw
    log_path,
Exception: Could not initialize logging: Error opening pantsd logfile: Permission denied (os error 13)
👀 1
h

hundreds-father-404

12/01/2021, 4:34 PM
You should be able to run the PEX with system python so long as it's Py37-39. Use
python3.7 pants.2.8.1rc1.pex
for example, i.e. override the shebang
Sounds like those Pyenv-managed interpreters for some reason are severely restricted with their permissions? Their pip doesn't work, and opening a file isn't working either
Does using the PEX with
sudo
work?
b

brash-baker-91190

12/01/2021, 4:39 PM
yeah,
sudo
works
h

hundreds-father-404

12/01/2021, 4:48 PM
K, is that an acceptable workaround? Also I'm curious if simply doing
pip install pytest
for example works using a Pyenv interpreter. It sounds like this isn't a Pants problem in particular, rather that Pants is trying to install itself with an interpreter that has insufficient permissions. I think that test would prove that
b

brash-baker-91190

12/01/2021, 4:52 PM
pip install pytest
works fine, actually 🤔
👀 1
I'd rather not use
sudo
if I can help it... I may need to do some more asking around on the Gitpod side of things
👍 1
h

hundreds-father-404

12/01/2021, 4:55 PM
Makes sense. I think continuing to narrow down the simplest reproduction might help with Gitpod support - looks like running the
pip install -U pip
command is failing (one thing to be sure of is you're using the correct Pyenv pip, not the system interpreter)
b

brash-baker-91190

12/01/2021, 4:56 PM
The command that appears to be failing is:
Copy code
/home/gitpod/.cache/pants/setup/bootstrap-Linux-x86_64/pants.7zPr2Q/install/bin/pip install -U pip
h

hundreds-father-404

12/01/2021, 4:57 PM
That can probably be simplified further. The bash script is simply creating a virtualenv, which is what results in
.cache/pants/setup/bootstrap-Linux-x86_64/pants.7zPr2Q/install/bin/pip
Instead, try running that command with the original
pip
corresponding to the interpreter used to create the venv
b

brash-baker-91190

12/01/2021, 5:02 PM
sorry, I'm not quite following
So, apparently this was happening because Gitpod is setting
PIP_USER=yes
in the environment, which is apparently a way to make every
pip
command behave as though
--user
was passed. Once I unset that (or, alternatively, set it to
no
), Pants works fine.
1
👀 1
h

hundreds-father-404

12/01/2021, 10:27 PM
Yay, great find!
b

brash-baker-91190

12/01/2021, 10:27 PM
Also, the "permission denied" errors I had earlier were a side effect of having tried running
pants
using
sudo
... the
.pants.d
and
.pids
directories were owned by
root
, and wackiness ensued 😅
1
h

hundreds-father-404

12/01/2021, 10:28 PM
Oh interesting!! Thanks for that update
So in the future, could have probably run the
pip
command inside the
./pants
bash script with
sudo
, then keep the final
exec
statement not-sudo. Although your fix is much better
b

brash-baker-91190

12/01/2021, 10:33 PM
The
sudo
was also a bit of a red-herring... it didn't work because it had magic
root
powers, it worked because
PIP_USER
wasn't in its environment. I just tried running
sudo -E ./pants version
(after blowing away all Pants cache directories) and it failed with the same message.
👍 1
I guess a fix might be to run these commands in a subshell with
PIP_USER
unset. Looks like you all do something similar with the
scrub_PEX_env_vars
function in the
pants
shell script.
That seems like it might actually do the trick... I just added
unset PIP_USER
at the end of
scrub_PEX_env_vars
and it worked. Probably need to kick the tires on some other more complicated projects (not sure if other things need to
pip install
into that venv or not).
🙌 1
It looks like another possibility that is perhaps more robust would be to add the
--isolated
flag to the two
pip
invocations in the
pants
script: https://pip.pypa.io/en/stable/cli/pip/#cmdoption-isolated
I'm no Python expert, though, so I'm not quite sure which approach is "the right one" (or if there's another way that's better)
h

hundreds-father-404

12/01/2021, 11:04 PM
Hm based on my read of https://stackoverflow.com/a/42989020, I'm not sure we would want to hardcode either approach in the
./pants
script we distribute. I can imagine cases where you do want
--user
to be used or for env vars like Certs to be still used Instead, probably better to have your version of the script diverge from the one provided w/ that one line. Wdyt? And even better is distribute Pants as a binary https://github.com/pantsbuild/pants/issues/7369
b

brash-baker-91190

12/01/2021, 11:15 PM
I can also work around it without messing with our
pants
scripts... Gitpod allows you to customize the container it uses for your workspace, so I just tweaked ours to remove that one variable. If we end up adopting Gitpod more broadly, I imagine we'd just have a base image that all our repos would use, and they'd all inherit that configuration. Modifying our
pants
script would get weird when we need to update it to pull in changes that you all make to it, so I'd rather steer clear of that if at all possible. Distributing Pants as a Rust binary looks pretty sweet, though 😄
💯 1
👍 1
h

hundreds-father-404

12/01/2021, 11:20 PM
So many Rust changes we want to make 😄 We're thinking about writing a blog post about our recent migration from Rust-CPython to PyO3 for our Rust FFI also
b

brash-baker-91190

12/01/2021, 11:22 PM
We'd be very interested to read that blog post... we're looking at PyO3 for our stuff, too
🚀 1
h

hundreds-father-404

12/01/2021, 11:34 PM
Oh sweet! PyO3 has been amazing, we're very happy with it and its community
b

brash-baker-91190

12/01/2021, 11:41 PM
nice to hear!
6 Views