I've been trying to introduce

torch-geometric

into my monorepo, and I've come across an interesting problem. Installing this depends on

torch-scatter

and

torch-sparse

(docs here), but the issues arises here with M1 macbooks. There are no official arm64 wheels available on the index. This is usually fine - i can install from source, but

setup.py

on these packages have a hard dependency on

torch

(link), so trying to do this would lead to

module not found: torch

. I followed the docs for undeclared dependencies, https://www.pantsbuild.org/docs/python-third-party-dependencies#requirements-with-undeclared-dependencies, but this still generates an issue with

python setup.py egg_info

, stack trace:

WARNING: Discarding git+<https://github.com/rusty1s/pytorch_scatter@62a958f467bf80c3f566083b4949ec21e2be57d0>. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: Discarding git+<https://github.com/rusty1s/pytorch_sparse@6143af2112cd404adca42111daf14397a45f2d7e>. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement torch-scatter (unavailable)
ERROR: No matching distribution found for torch-scatter (unavailable)

Pinning a single page link source is also problematic since it should build for setups with cuda & without cuda on cpu - there are single page source for each of these cases for the wheels https://data.pyg.org/whl/torch-1.11.0+cpu.html and https://data.pyg.org/whl/torch-1.11.0+cu113.html Any tips/ideas on how to get around this issue? (This usually is no problem in conda because of the package metadata, but there's no way to incorporate conda into pants right?)

Is there a way to move files at runtime outside of the

BUILD

directory (or subdirectories)? I'm using Voila to run a notebook, and it is expecting files to be in specific places on the file system. I tried using

files

and

relocated_files

targets, but you can't throw an absolute path as a destination.

Is the the

dependees

goal supposed to respect the

--filter-target-types

option? It does not seem like it does to me.

When a pex build gets stuck (spinning on

Building N requirements for Y.pex

), is there a way to interrogate what requirement is jamming it up? It seems like it'll happily "build" forever without making any progress.

I use to do a

ps ax | grep pants

which will show you running processes and one of them likely has a tmp path with interesting log files to tail during a run… 😉

notably the

pex

tools invocation of

pip

has a pip log file arg on the command line worth digging up 🙂

and if it actually finishes, you can use

--pex-verbosity=3 -ldebug

or higher to get some more info on where time was being spent, i think?

Well it'll definitely finish once I ctrl+c it, so I can try that next time!

Has anyone tried to run a pex from a distroless container?

Hi all! I'm getting the following error when running

./pants package

on a

docker_image

target:

#3 [internal] load metadata for <http://docker.io/library/python:3.9.10|docker.io/library/python:3.9.10>
#3 sha256:46aa02c10fee7f9ebc6cc466be0c44172ff37ff4dd662dc045c92c93ba51e3fe
#3 ERROR: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: `One or more parameters passed to a function were not valid.`

has anyone ran into this issue?

searching

One or more parameters passed to a function were not valid

looks like it has something to do with ssl certificates in macs

weird thing is that using docker directly works totally fine

nothing I recognize, but seems to be related to a docker auth plugin. Does it work when you run docker outside of pants? If so, it’s likely some missing env var/PATH needed for the pants docker configuration that needs to be set in order for the auth plugin to work.

it can be a bit of a pain to track down/identify precisely what is required..

thank you!!

adding

[docker]
env_vars = ["DOCKER_CONFIG=%(homedir)s/.docker"]

to

pants.toml

solved the issue

weird that I have to do this though, since that's the default location

Then maybe the notion of the current user/home dir is not specified when running Docker from pants..

But happy you were able to solve it 👍

if neither

XDG_CONFIG_HOME

nor

HOME

is not set in the execution sandbox, docker would not know where to find config file. https://github.com/moby/moby/blob/2400bc66ef6955760a331a2bff767d4fec244844/pkg/homedir/homedir_linux.go#L84-L93

How can I set arbitrary pex parameters - e.g. PEX_ROOT - for all times pants runs pex?

If you're ever curious to learn more about sandboxing in Pants, this is a good intro talk about that

How do I get the standalone test runner pex that would re-run the test goal that went into it creating it, so that an outsider could re-run failed integration tests without having the workspace from which the pants CLI was used? Does such an export option exist?

I'm having a lot of recurring issues with 3rd party dependencies. So before proceeding with a question for each issue, I want to start by clarifying what the "proper" way to use

python_requirement

and

python_requirements

is, in case there's some misuse going on that's messing things up.

I'd like to get a glimpse on when pants & pex is going to support custom entrypoints (other than console scripts) (https://github.com/pantsbuild/pants/issues/11386), as this is one of the remaining pieces to completely migrate our workflows. As some of you know, I've been workarounding this for a while using a custom entrypoint loader that scans BUILD files for development setups, but in this way I cannot still exploit the convenience of pex packages for distributing in customer sites. (e.g., there are cases that we cannot modify the target virtual environment but it becomes much feasible if I could mount/inject some files or directories inside immutable containers.) If it's possible, I'm also considering direct code contributions, but I'm afraid that it would require a bit deeper understanding of pex and pants...

I’m trying to enable coverage for python with pants however I’m stuck. 1. When I set

use_coverage = true

in

pants.toml

it complains:

[pytest].extra_requirements is missing pytest-cov, which is needed to collect coverage data.

2. Since

pytest-cov

is a new

extra_requirement

for

pytest

, I have to run

./pants generate-lockfiles --resolve=pytest

. Otherwise, it complains with

InvalidLockfileError

. 3. However, when I run

./pants generate-lockfiles --resolve=pytest

, pants does not generate hashes for some dependencies. 4. So when I run

./pants test ::

it starts to complain:

ProcessExecutionFailure: Process 'Building pytest.pex from src/python/tests/lockfile_pytest.txt' failed with exit code 1.
stdout:

stderr:
ERROR: Hashes are required in --require-hashes mode, but they are missing from some requirements. Here is a list of those requirements along with the hashes their downloaded archives actually had. Add lines like these to your requirements files to prevent tampering. (If you did not enable --require-hashes manually, note that it turns on automatically when any package has a hash.)

any ideas?

What version of pants are you using? I didn’t have to add any extra requirement for enabling coverage.

@refined-addition-53644 I suspect that's because you're using the default pants-provided lockfile for pytest, which includes it. Otherwise if you use your own, you have to also add the additional requirements if you want them.

The “pants does not generate hashes for some dependencies.” sounds like a bug

I don’t reproduce this at HEAD