how does one view the errors associated with the generate-lockfile goal for pants? I've got an issue where the generation fails with the following error:

pid 75651 -> /Users/jmello/sandbox/pants-build/named_caches/pex_root/venvs/a80988bb3f0e73c2683977f47507e857bda85f08/b1226bdc6f65481853a9719d7912127489403612/bin/python -sE /Users/jmello/sandbox/pants-build/named_caches/pex_root/venvs/a80988bb3f0e73c2683977f47507e857bda85f08/b1226bdc6f65481853a9719d7912127489403612/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/jmello/sandbox/pants-build/named_caches/pex_root/pip_cache --log /private/var/folders/6x/764jvn315hg0clblwyvqy2lrvmwrk7/T/pants-sandbox-gioMjM/.tmp/pex-pip-log.p96kty96/pip.log download --dest /private/var/folders/6x/764jvn315hg0clblwyvqy2lrvmwrk7/T/pants-sandbox-gioMjM/.tmp/tmpgjgv70hm/Users.jmello..pyenv.versions.3.10.3.bin.python3.10 --constraint constraints.txt dkpro-cassis==0.2.1 joblib<2.0.0 numpy<1.25.0 scikit-learn==0.22.2.post1 setuptools>=41.0.0 spacy==2.1.9 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

I can't seem to figure out how to get the actual log of the build/lockfile generation to see exactly what was happening. I've got the following files that allow reproduction of this issue. For completeness this is on a Mac x86 system. The spacy, thinc, and a couple of other wheels do not have binaries built for this platform, so they will have there deps extracted from an sdist: BUILD:

python_requirements(
    name = "requirements",
    resolve = "event_detection"
)

pants.toml

[GLOBAL]
pants_version = "2.17.0"
pants_workdir = "%(buildroot)s/pants-build/.pants.d"
pants_subprocessdir= "%(buildroot)s/pants-build/.pids"
local_store_dir = "%(buildroot)s/pants-build/lmdb_store"
named_caches_dir = "%(buildroot)s/pants-build/named_caches"
backend_packages = [
    "pants.backend.python",
    "pants.backend.project_info",
    "pants.backend.python.lint.pylint",
    "pants.backend.shell",
    "pants.backend.docker"
]

[python]
interpreter_constraints = [ ">=3.8" ]
default_resolve = "python_default"
enable_resolves = true

[python.resolves]
event_detection = "event-detection.lock"

[python.resolves_to_constraints_file]
event_detection = "constraints.txt"

[python-infer]
use_rust_parser = true

[setup-py-generation]
first_party_dependency_version_scheme = "compatible"

[anonymous-telemetry]
enabled = false

requirements.txt:

dkpro-cassis==0.2.1
numpy<1.25.0
joblib<2.0.0
setuptools>=41.0.0
scikit-learn==0.22.2.post1
spacy==2.1.9

constraints.txt:

wheel>0.32.0,<0.33.0
cymem>=2.0.2,<2.1.0
preshed>=2.0.1,<2.1.0
murmurhash>=0.28.0,<1.1.0
thinc>=7.0.8,<7.1.0

not ideal but sometimes there are homebrew packages required to install some python packages. if you have output beyond what you copied and pasted you might be able to see the root cause

I can't find a way to get additional output unfortunately. I can keep the sandbox and execute the

__run.sh

but I only get that output.

my guess is joblib or scikit learn is the culprit

probably, but I'm at a loss to diagnose.

try running with

PEX_VERBOSE=9

or so may be illustrative.. see this related thread for more tips & tricks https://pantsbuild.slack.com/archives/C046T6T9U/p1693392344130759

@curved-television-6568 Thanks, getting more info on the build failure. It is interesting as the pex generate-lockfile script is trying to do an actual build which is failing on MacOSX. I have a local_environment defined like thus, but the

generate-lockfiles

target seems to ignore the fallback:

docker_environment(
    name = "centos7_python38",
    image = "<http://racp-dev.docker.artifactory.rii.io/python38-builder-image:1.0.0|racp-dev.docker.artifactory.rii.io/python38-builder-image:1.0.0>",
    platform = "linux_x86_64"
)

local_environment(
    name = "local_environment",
    compatible_platforms=["linux_x86_64"],
    fallback_environment = "centos7_python38"
)

and pants.toml:

[environments-preview.names]
local_environment = "//:local_environment"
centos7_python38 = "//:centos7_python38"

Looking at the code the generate-lockfiles goal should be selecting

__local__

as it's environment, which I would have assumed would spin up the appropriate docker environment with all the required goodies.

@blue-city-97042 re:

Also, I would hope that it would be able to glean the dependencies for the package without having to do a binary install.

You may have missed:

pip subprocess to install build dependencies did not run successfully.

So these are not runtime dependencies, but build dependencies. If you read through PEP-517 / PEP-518, to even be able to determine final artifact metadata, Pip must be able to load the build backend in question in order to ask it what the artifact metadata is. It's failing to do that. Now, this may be because

[build-system] requires = [...]

include more than just the build backend requirements or it may be because its failing to install the results of

get_requires_for_build_wheel

which is required before calling

prepare_metadata_for_build_wheel

(or

build_wheel

) per the spec: https://peps.python.org/pep-0517/#get-requires-for-build-wheel.

regarding environments, are you on a arm Mac? (M1 or M2) in that case your

docker_environment

is not compatible with the specified platform, I think..

x86 MacOS

ok, then I don’t know about the environments.. 🤷

@enough-analyst-54434 The error is occurring when trying to build bliss, which needs to be compiled. But unless I can specify a docker environment to run things on, it'll be difficult to get a local developer environment setup with all of the python libs, compilers and other tools.

@blue-city-97042 gotcha on all the Pants level concerns. I'm the Pex maintainer / Pip guy; so I just wanted to clear up your confusion there if you had any.

yeah, it's killing me as the rest of the project stuff has been smooth sailing. doing dependency locks on stuff that doesn't have binary support for our platform has been difficult at best. I can do naked builds using pip, but it involves a lot of magic I never expected to have to know.

I strongly suspect that in T-2.5 years, Python will have had its lunch ate in ML due to not having addressed Python packaging. All signs is the problems won't be close to solved in that time, they have talked at length about their (PyPA) own solicited developer feedback and proceeded to get ~nowhere towards addressing what the overwhelming sentiment was: wrap this stuff up in Python / PyPA so there is 1 tool, 1 way to do things, it works.

@enough-analyst-54434 do you have any idea why my local_environment is getting ignored? I'm trying to trace through code, but I'm unsure of the handoff between the GenerateLockFiles class and the subsystem.

I have no clue about Pants stuff for the most part.

oh, np then 🙂

I have some sense of pants stuff, but very little when it comes to the environments feature. I can’t see that the fallback is considered for the local environment for instance, looking at this rule: https://github.com/pantsbuild/pants/blob/60fcdf54695ca85b4cf1e29f88187c5d83a295ca/src/python/pants/core/util_rules/environments.py#L606-L619

that seems broken then....

yea, I agree, the docs seems to suggest this is supposed to work. or I’m just not seeing it..

hm, yea: that’s unexpected. @blue-city-97042: would you mind filing an issue with some of the steps involved in reproducing?

that's for the local_environment issue, correct?

yes

@witty-crayon-22786 https://github.com/pantsbuild/pants/issues/19872