https://pantsbuild.org/ logo
Join Slack
Powered by
Hi, I'm evaluating if Pants would be correct choic...
# general
e
Hi, I'm evaluating if Pants would be correct choice for our project. I'm trying 
pants
2.16.0 on a very simple dummy python project, inside a virtual environment and getting
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants roots
11:47:04.85 [INFO] Initialization options changed: reinitializing scheduler...
11:47:09.70 [INFO] Scheduler initialized.
11:47:09.75 [ERROR] 1 Exception encountered:

Engine traceback:
  in `roots` goal
  in Compute all source roots

IntrinsicError: Failed to read link "/home/niko/tmp/pants-tests/venv/bin/python": Absolute symlink: "/usr/bin/python"
Any idea what would be wrong? In addition, is this the correct place to ask help or is there some other preferred place for this kind of questions?
✅ 1
r
Yes it's the correct place to ask such questions. Can you share your 
pants.toml
please? What OS are you on?
e
I'm glad to hear I'm in the right place. The 
pants.toml
is very minimal. Simply
Copy code
[GLOBAL]
pants_version = "2.16.0"
I also tried:
Copy code
[GLOBAL]
pants_version = "2.16.0"

[source]
root_patterns = ['/']
and
Copy code
[GLOBAL]
pants_version = "2.16.0"
backend_packages = [
  "pants.backend.python"
]

[source]
root_patterns = ['/']
This is the project structure:
Copy code
pants-tests/
├─setup.py
├─pants.toml
└─mypkg/
  ├─__init__.py
  └─something.py
I am running the commands in 
pants-tests
folder. The 
venv
is also located in that folder.
r
does 
pants version
show anything?
e
Yes, it prints out the version:
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants version
2.16.0
r
ok, that's a good start. Can you run with the python backend enabled?
Copy code
pants tailor ::
e
I missed the OS question. I'm running on Ubuntu 22.04.2 LTS
👍 1
I ran 
pants tailor ::
with 
backend_packages = ["pants.backend.python"]
in 
pants.toml
and it looks similar to the 
pants roots
output
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants tailor ::
12:46:50.97 [INFO] waiting for pantsd to start...
12:46:52.07 [INFO] pantsd started
12:46:52.13 [INFO] Initializing scheduler...
12:46:56.93 [INFO] Scheduler initialized.
12:46:56.99 [ERROR] 1 Exception encountered:

Engine traceback:
  in `tailor` goal

IntrinsicError: Failed to read link "/home/niko/tmp/pants-tests/venv/bin/python": Absolute symlink: "/usr/bin/python"
r
Are you able to use this virtual env without pants? I would first try to run some python script inside this virtualenv.
e
The python executable in the virtual env seems to work fine. For example, adding to the `something.py`:
Copy code
...
if __name__ == "__main__":
    print("Hello from ", __file__)
Gives:
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ python mypkg/something.py 
Hello from  /home/niko/tmp/pants-tests/mypkg/something.py
I can also verify the sys.executable:
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ python
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.executable)
/home/niko/tmp/pants-tests/venv/bin/python
I have to say, I got 
pants roots
and 
pants tailor
working without errors in another project structure earlier, but faced problems trying to configure it correctly, so I decided to start with a small dummy project.
r
And did you run all the pants command after activating the virtual env? Try nuking the cache 
~/.cache/nce/
and running 
pants version
again to set-it up after activating this virtual env
e
Good question. I can't remember. I'll try that!
r
Just to be safe, if the 
~/.cache/pants
exist, remove that too.
e
Just to verify: You're not meaning to delete anything in the project folder:
Copy code
niko@niko-ubuntu:~/tmp/pants-tests$ ls -la
total 36
drwxrwxr-x  7 niko niko 4096 elo    11 11:31 .
drwxrwxr-x 14 niko niko 4096 elo    11 12:40 ..
drwxrwxr-x  2 niko niko 4096 elo    11 11:31 my_cool_pkg.egg-info
drwxrwxr-x  2 niko niko 4096 elo    11 10:09 mypkg
drwxrwxr-x  3 niko niko 4096 elo    11 12:46 .pants.d
-rw-rw-r--  1 niko niko  110 elo    11 12:46 pants.toml
drwxrwxr-x  3 niko niko 4096 elo    11 12:53 .pids
-rw-rw-r--  1 niko niko  157 elo    11 10:57 setup.py
drwxrwxr-x  6 niko niko 4096 elo    11 12:53 venv
but in the user home 
~/
?
r
yes
✔️ 1
e
I first removed 
~/.cache/pants
and 
~/.cache/nce
and then did activate the venv again and ran 
pants version
and tried 
pants roots
. Here's the output.
Copy code
niko@niko-ubuntu:~/tmp/pants-tests$ source venv/bin/activate
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants version
Bootstrapping Pants 2.16.0 using cpython 3.9.16
Installing pantsbuild.pants==2.16.0 into a virtual environment at /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.16.0
New virtual environment successfully created at /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.16.0.
2.16.0
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants roots
13:02:54.59 [ERROR] 1 Exception encountered:

Engine traceback:
  in `roots` goal
  in Compute all source roots

IntrinsicError: Failed to read link "/home/niko/tmp/pants-tests/venv/bin/python": Absolute symlink: "/usr/bin/python"
meanwhile the 
pants roots
command seems to work fine in an another project on the same machine.
I got it working by changing the project structure!
This does not work project structure:
Copy code
📁 pants-tests/
├─📄 setup.py
├─📄 pants.toml
└─📁 mypkg/
  ├─📄 __init__.py
  └─📄 something.py
pants.toml
Copy code
[GLOBAL]
pants_version = "2.16.0"
backend_packages = ["pants.backend.python"]

[source]
root_patterns = ['/']
This works (put everything under 
./src
) project structure:
Copy code
📁 pants-tests/
├─📁 src/
│ ├─📄 setup.py
│ └─📁 mypkg/
└─📄 pants.toml
pants.toml
Copy code
[GLOBAL]
pants_version = "2.16.0"
backend_packages = ["pants.backend.python"]

[source]
root_patterns = ['/src']
Or, I'm not sure if it works. I'm getting now with 
pants roots
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants roots
src
but with 
pants tailor
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants tailor ::
13:13:19.02 [ERROR] 1 Exception encountered:

Engine traceback:
  in `tailor` goal

IntrinsicError: Failed to read link "/home/niko/tmp/pants-tests/venv/bin/python": Absolute symlink: "/usr/bin/python"
I wonder if there's any possibility to get more detailed error messages or see some type of logs?
r
yeah quite cryptic to be honest. I am not sure what's going on either
e
Thanks anyway @gentle-florist-96289 ! I could also try an older version of 
pants
r
Yeah worth trying 2.15.0
e
Is that as simple as changing 
pants.toml
to have
Copy code
pants_version = "2.15.0"
and running a command?
r
Is that as simple as changing 
pants.toml
to have
yes As far as debugging go, you can check this
Copy code
pants --print-stacktrace -ldebug <rest of your command>
https://www.pantsbuild.org/docs/troubleshooting
e
Seems that the pants 2.15.0 works similarly. Here's the stacktrace:
Copy code
(venv) niko@niko-ubuntu:~/tmp/pants-tests$ pants --print-stacktrace -ldebug tailor ::
13:29:09.01 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7fc9b435eca0>
13:29:09.01 [DEBUG] terminating pantsd
13:29:09.01 [DEBUG] sending signal 15 to pid 158713
13:29:09.21 [DEBUG] successfully terminated pid 158713
13:29:09.21 [DEBUG] purging metadata directory: /home/niko/tmp/pants-tests/.pids/6bfb3ce52fe9/pantsd
13:29:09.21 [DEBUG] Launching pantsd
13:29:09.21 [DEBUG] purging metadata directory: /home/niko/tmp/pants-tests/.pids/6bfb3ce52fe9/pantsd
13:29:09.22 [DEBUG] pantsd command is: PANTS_DAEMON_ENTRYPOINT=pants.pantsd.pants_daemon:launch_new_pantsd_instance PYTHONPATH=/home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/bin:/home/niko/code/omat:/home/niko/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python39.zip:/home/niko/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9:/home/niko/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/lib-dynload:/home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/lib/python3.9/site-packages /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/bin/python /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/bin/pants --python-repos-find-links=-['file:///home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/find_links/2.15.0/214abc14/index.html'] --print-stacktrace -ldebug tailor ::
13:29:10.72 [DEBUG] pantsd is running at pid 158773, pailgun port is 32933
13:29:10.72 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7fc9b435eca0>
13:29:10.72 [DEBUG] Connecting to pantsd on port 32933
13:29:10.72 [DEBUG] Connecting to pantsd on port 32933 attempt 1/3
13:29:10.72 [DEBUG] Connected to pantsd
13:29:10.73 [DEBUG] Launching 1 roots (poll=false).
13:29:10.75 [DEBUG] computed 1 nodes in 0.018775 seconds. there are 9 total nodes.
13:29:10.75 [DEBUG] Launching 1 roots (poll=false).
13:29:10.75 [DEBUG] computed 1 nodes in 0.000763 seconds. there are 13 total nodes.
13:29:10.78 [INFO] Initializing scheduler...
13:29:10.79 [DEBUG] File handle limit is: 10000
13:29:10.82 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }] for process execution.
13:29:11.23 [DEBUG] Changes to /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/bin, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] Changes to /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/bin, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] Changes to /home/niko/code/omat, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] Changes to /home/niko/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python39.zip, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] Changes to /home/niko/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] Changes to /home/niko/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/lib-dynload, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] Changes to /home/niko/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.15.0/lib/python3.9/site-packages, outside of the buildroot, will not be invalidated.
13:29:11.23 [DEBUG] setting up service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7f2c5001ce20>
13:29:11.23 [DEBUG] setting up service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7f2c500207c0>
13:29:11.23 [DEBUG] starting service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7f2c5001ce20>
13:29:11.23 [DEBUG] starting service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7f2c500207c0>
13:29:11.23 [INFO] Scheduler initialized.
13:29:11.23 [DEBUG] Launching 1 roots (poll=false).
13:29:11.23 [DEBUG] computed 1 nodes in 0.000531 seconds. there are 13 total nodes.
13:29:11.23 [DEBUG] Launching 1 roots (poll=false).
13:29:11.23 [DEBUG] computed 1 nodes in 0.000154 seconds. there are 13 total nodes.
13:29:11.26 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(RecursiveGlobSpec(directory=''),), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False))
13:29:11.26 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
13:29:11.26 [DEBUG] Launching 1 roots (poll=false).
13:29:11.28 [DEBUG] computed 1 nodes in 0.015743 seconds. there are 13 total nodes.
13:29:11.28 [DEBUG] Launching 1 roots (poll=false).
13:29:11.28 [WARN] Please either set `enabled = true` in the [anonymous-telemetry] section of pants.toml to enable sending anonymous stats to the Pants project to aid development, or set `enabled = false` to disable it. No telemetry sent for this run. An explicit setting will get rid of this message. See <https://www.pantsbuild.org/v2.15/docs/anonymous-telemetry> for details.
13:29:11.28 [DEBUG] computed 1 nodes in 0.001310 seconds. there are 21 total nodes.
13:29:11.28 [DEBUG] Launching 1 roots (poll=false).
13:29:11.28 [DEBUG] computed 1 nodes in 0.000205 seconds. there are 21 total nodes.
13:29:11.28 [DEBUG] requesting <class 'pants.core.goals.tailor.TailorGoal'> to satisfy execution of `tailor` goal
13:29:11.28 [DEBUG] Launching 1 roots (poll=false).
13:29:11.28 [DEBUG] Completed: Find all sources from input specs
13:29:11.29 [DEBUG] Completed: `tailor` goal
13:29:11.29 [DEBUG] computed 1 nodes in 0.007573 seconds. there are 113 total nodes.
13:29:11.29 [ERROR] 1 Exception encountered:

Engine traceback:
  in select
    ..
  in pants.core.goals.tailor.tailor
    `tailor` goal
  in pants.engine.internals.specs_rules.resolve_specs_paths
    Find all sources from input specs
  in paths
    Finding files: **

Traceback (no traceback):
  <pants native internals>
Exception: Failed to read link "/home/niko/tmp/pants-tests/venv/bin/python": Absolute symlink: "/usr/bin/python"
My only thoughts are, either 1. I have created the virtual environment and activated it in an order which 
pants
does not like. I could try also a fresh start. If so, what is the correct order of actions?
2. I see lots of links in 
usr/bin
related to python. Could that be the reason?
Copy code
niko@niko-ubuntu:/usr/bin$ ls -la  | grep python
lrwxrwxrwx  1 root root          23 heinä   1  2022 pdb2.7 -> ../lib/python2.7/pdb.py
lrwxrwxrwx  1 root root          24 kesä   11 08:26 pdb3.10 -> ../lib/python3.10/pdb.py
lrwxrwxrwx  1 root root          31 elo    18  2022 py3versions -> ../share/python3/py3versions.py
lrwxrwxrwx  1 root root           7 loka   11  2021 python -> python3
lrwxrwxrwx  1 root root           9 heinä  28  2021 python2 -> python2.7
-rwxr-xr-x  1 root root     3592504 heinä   1  2022 python2.7
lrwxrwxrwx  1 root root          10 elo    18  2022 python3 -> python3.10
-rwxr-xr-x  1 root root     5913032 kesä   11 08:26 python3.10
lrwxrwxrwx  1 root root          34 kesä   11 08:26 python3.10-config -> x86_64-linux-gnu-python3.10-config
lrwxrwxrwx  1 root root          17 elo    18  2022 python3-config -> python3.10-config
-rwxr-xr-x  1 root root         960 tammi  25  2023 python3-futurize
-rwxr-xr-x  1 root root         964 tammi  25  2023 python3-pasteurize
lrwxrwxrwx  1 root root          29 heinä  28  2021 pyversions -> ../share/python/pyversions.py
-rwxr-xr-x  1 root root        3123 kesä   11 08:26 x86_64-linux-gnu-python3.10-config
lrwxrwxrwx  1 root root          34 elo    18  2022 x86_64-linux-gnu-python3-config -> x86_64-linux-gnu-python3.10-config
although, #2 is probably not the reason as 
pants
commands work on the other project on the same machine.
b
Pants manages Venvs internally, so I’d suggest not using them when running pants. Does it work without the venv?
e
@broad-processor-92400 Oh I must have missed that while reading the Getting Started pages! I'll try without a venv!
👍 1
That was it
Copy code
niko@niko-ubuntu:~/tmp/pants-tests$ pants tailor ::
13:42:01.08 [INFO] Initializing scheduler...
13:42:01.51 [INFO] Scheduler initialized.
13:42:01.55 [WARN] Please either set `enabled = true` in the [anonymous-telemetry] section of pants.toml to enable sending anonymous stats to the Pants project to aid development, or set `enabled = false` to disable it. No telemetry sent for this run. An explicit setting will get rid of this message. See <https://www.pantsbuild.org/v2.15/docs/anonymous-telemetry> for details.
Created src/BUILD:
  - Add python_sources target src
Created src/mypkg/BUILD:
  - Add python_sources target mypkg
🎉 1
b
Sorry for the fuss. When I’m next at a computer, I’ll file a bug to improve the diagnostics, because this seems like a common way to trip up that could easily be detected/explained by pants itself rather than needing external help
🙏 1
h
@enough-advantage-61561, can you elaborate on what you mean by "created the virtual environment"? Did you install Pants via 
pip install pantsbuild.pants==2.16.0
or something like that?
e
I created virtual environment with 
python -m venv venv
and used it. That was before I came to know that Pants does not work (or should not be used) inside virtual environment. I installed Pants using
Copy code
curl --proto '=https' --tlsv1.2 -fsSL <https://static.pantsbuild.org/setup/get-pants.sh> | bash
And by "inside virtual environment", I mean, while a virtual environment is activated in current shell.
h
Ah, makes sense
Although, I would like that to work, since 
get-pants.sh
should be isolated from any pythons on your PATH
I don't think you were doing anything wrong, so I would consider this a bug in our bootstrapping mechanism
even if the remedy is easy
e
Just some helpful exception would save a ton of time 😄 Or some really big info/warning text in the Getting Started pages.
b
2 Views
Open in Slack
PreviousNext
Powered by