https://pantsbuild.org/ logo
Join Slack
Powered by
Has anybody had experience running Pants on <gitpo...
# general
b
Has anybody had experience running Pants on gitpod.io? I'm running into 
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
when I try to run anything (even 
./pants version
)
👀 1
h
Hey Chris, I suspect this might be when installing Pants via the bash script? Could you please add 
set -x
to the top and try running again, copy the output if posible here?
b
h
Copy code
++ /home/gitpod/.cache/pants/setup/bootstrap-Linux-x86_64/pants.8SS4Vl/install/bin/pip install -U pip
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
WARNING: You are using pip version 21.1.2; however, version 21.3.1 is available.
You should consider upgrading via the '/home/gitpod/.cache/pants/setup/bootstrap-Linux-x86_64/pants.8SS4Vl/install/bin/python -m pip install --upgrade pip' command.
b
This is using Gitpod's base container image, which appears to use Pyenv for its Python environments. I'm able to run things under 
sudo
and have it work, which ends up using a system Python (not Pyenv), if that's useful.
h
b
Initially got this error when running from the PEX:
Copy code
/usr/bin/env: 'python3.7': No such file or directory
Installing a 3.7 Python now....
After installing and enabling a 3.7 Python...
Copy code
$ pyenv versions
  system
* 3.7.0 (set by /workspace/cloudsmith-buildkite-plugin/.python-version)
  3.8.12
I get the following error:
Copy code
$ ./pants.2.8.1rc1.pex version
Traceback (most recent call last):
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 476, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 532, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 668, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/home/gitpod/.pex/unzipped_pexes/2af639b86a62016e1736c9a6647e6a01ba5009f6/.bootstrap/pex/pex.py", line 700, in execute_pkg_resources
    return runner()
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_loader.py", line 115, in main
    PantsLoader.main()
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_loader.py", line 111, in main
    cls.run_default_entrypoint()
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_loader.py", line 93, in run_default_entrypoint
    exit_code = runner.run(start_time)
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_runner.py", line 77, in run
    with initialize_stdio(global_bootstrap_options), stdio_destination(
  File "/home/gitpod/.pyenv/versions/3.7.0/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/init/logging.py", line 167, in initialize_stdio
    global_bootstrap_options.pants_workdir,
  File "/home/gitpod/.pyenv/versions/3.7.0/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/gitpod/.pex/installed_wheels/3277bf65a33f7b4df0d47854d16a2eaf10bcfc1c/pantsbuild.pants-2.8.1rc1+git35901a6f-cp37-cp37m-linux_x86_64.whl/pants/init/logging.py", line 204, in initialize_stdio_raw
    log_path,
Exception: Could not initialize logging: Error opening pantsd logfile: Permission denied (os error 13)
👀 1
h
You should be able to run the PEX with system python so long as it's Py37-39. Use 
python3.7 pants.2.8.1rc1.pex
for example, i.e. override the shebang
Sounds like those Pyenv-managed interpreters for some reason are severely restricted with their permissions? Their pip doesn't work, and opening a file isn't working either
Does using the PEX with 
sudo
work?
b
yeah, 
sudo
works
h
K, is that an acceptable workaround? Also I'm curious if simply doing 
pip install pytest
for example works using a Pyenv interpreter. It sounds like this isn't a Pants problem in particular, rather that Pants is trying to install itself with an interpreter that has insufficient permissions. I think that test would prove that
b
pip install pytest
works fine, actually 🤔
👀 1
I'd rather not use 
sudo
if I can help it... I may need to do some more asking around on the Gitpod side of things
👍 1
h
Makes sense. I think continuing to narrow down the simplest reproduction might help with Gitpod support - looks like running the 
pip install -U pip
command is failing (one thing to be sure of is you're using the correct Pyenv pip, not the system interpreter)
b
The command that appears to be failing is:
Copy code
/home/gitpod/.cache/pants/setup/bootstrap-Linux-x86_64/pants.7zPr2Q/install/bin/pip install -U pip
h
That can probably be simplified further. The bash script is simply creating a virtualenv, which is what results in 
.cache/pants/setup/bootstrap-Linux-x86_64/pants.7zPr2Q/install/bin/pip
Instead, try running that command with the original 
pip
corresponding to the interpreter used to create the venv
b
sorry, I'm not quite following
So, apparently this was happening because Gitpod is setting 
PIP_USER=yes
in the environment, which is apparently a way to make every 
pip
command behave as though 
--user
was passed. Once I unset that (or, alternatively, set it to 
no
), Pants works fine.
❗ 1
👀 1
h
Yay, great find!
b
Also, the "permission denied" errors I had earlier were a side effect of having tried running 
pants
using 
sudo
... the 
.pants.d
and 
.pids
directories were owned by 
root
, and wackiness ensued 😅
❗ 1
h
Oh interesting!! Thanks for that update
So in the future, could have probably run the 
pip
command inside the 
./pants
bash script with 
sudo
, then keep the final 
exec
statement not-sudo. Although your fix is much better
b
The 
sudo
was also a bit of a red-herring... it didn't work because it had magic 
root
powers, it worked because 
PIP_USER
wasn't in its environment. I just tried running 
sudo -E ./pants version
(after blowing away all Pants cache directories) and it failed with the same message.
👍 1
I guess a fix might be to run these commands in a subshell with 
PIP_USER
unset. Looks like you all do something similar with the 
scrub_PEX_env_vars
function in the 
pants
shell script.
That seems like it might actually do the trick... I just added 
unset PIP_USER
at the end of 
scrub_PEX_env_vars
and it worked. Probably need to kick the tires on some other more complicated projects (not sure if other things need to 
pip install
into that venv or not).
🙌 1
It looks like another possibility that is perhaps more robust would be to add the 
--isolated
flag to the two 
pip
invocations in the 
pants
script: https://pip.pypa.io/en/stable/cli/pip/#cmdoption-isolated
I'm no Python expert, though, so I'm not quite sure which approach is "the right one" (or if there's another way that's better)
h
Hm based on my read of https://stackoverflow.com/a/42989020, I'm not sure we would want to hardcode either approach in the 
./pants
script we distribute. I can imagine cases where you do want 
--user
to be used or for env vars like Certs to be still used Instead, probably better to have your version of the script diverge from the one provided w/ that one line. Wdyt? And even better is distribute Pants as a binary https://github.com/pantsbuild/pants/issues/7369
b
I can also work around it without messing with our 
pants
scripts... Gitpod allows you to customize the container it uses for your workspace, so I just tweaked ours to remove that one variable. If we end up adopting Gitpod more broadly, I imagine we'd just have a base image that all our repos would use, and they'd all inherit that configuration. Modifying our 
pants
script would get weird when we need to update it to pull in changes that you all make to it, so I'd rather steer clear of that if at all possible. Distributing Pants as a Rust binary looks pretty sweet, though 😄
💯 1
👍 1
h
So many Rust changes we want to make 😄 We're thinking about writing a blog post about our recent migration from Rust-CPython to PyO3 for our Rust FFI also
b
We'd be very interested to read that blog post... we're looking at PyO3 for our stuff, too
🚀 1
h
Oh sweet! PyO3 has been amazing, we're very happy with it and its community
b
nice to hear!
6 Views
Open in Slack
PreviousNext
Powered by