I'm having issues installing scie-pants for usage in a vscode devcontainer. Did something change in the versions list?

vscode ➜ /workspaces/python (main) $ pants
Bootstrapping Pants 2.15.0 using cpython 3.9.15
Installing pantsbuild.pants==2.15.0 into a virtual environment at /home/vscode/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/bindings/venvs/2.15.0
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 22.1 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.3/819.3 KB 41.9 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.5/64.5 KB 27.5 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.15.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 2.16.0.dev5, 2.16.0.dev6, 2.16.0.dev7, 2.16.0a0, 2.16.0a1, 2.16.0rc0, 2.17.0.dev0, 2.17.0.dev1, 2.17.0.dev2)
ERROR: No matching distribution found for pantsbuild.pants==2.15.0
Install failed: Command '['/home/vscode/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/bindings/venvs/2.15.0/bin/python', '-sE', '-m', 'pip', '--disable-pip-version-check', '--no-python-version-warning', '--log', PosixPath('/home/vscode/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/bindings/venvs/2.15.0/pants-install.log'), 'install', '--quiet', '--find-links', 'file:///home/vscode/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/bindings/find_links/2.15.0/214abc14/index.html', '--progress-bar', 'off', 'pantsbuild.pants==2.15.0']' returned non-zero exit status 1.
More information can be found in the log at: /home/vscode/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/bindings/logs/install.log

Error: Isolates your Pants from the elements.

Please select from the following boot commands:

<default>: Detects the current Pants installation and launches it.
bootstrap-tools: Introspection tools for the Pants bootstrap process.
pants: Runs a hermetic Pants installation.
pants-debug: Runs a hermetic Pants installation with a debug server for debugging Pants code.
update: Update scie-pants.

You can select a boot command by passing it as the 1st argument or else by setting the SCIE_BOOT environment variable.

ERROR: Failed to establish atomic directory /home/vscode/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/locks/install-c754e98484a5cbe1bd4d4ad6aeaf68a8e2937d10f0a11c2645d2c5bcb4212a3e. Population of work directory failed: Boot binding command failed: exit status: 1

I'm seeing something surprising when moving from 2.15.0 to 2.16.0rc0. I've moved my tool version management over from specifying

lockfile = lockfiles/foo.lock

for each tool before, to now having an

install_from_resolve = "build-tools"

, as suggested in the 2.18.x deprecation notice. I've written a

requirements.txt

with my tools versions in it, run

pants generate-lockfiles

, and I can see the lockfile is created correctly. However, when I actually try to use the tools (

pants fmt fix ::

), I get a failure about python version constraints which doesn't quite make sense to me:

...
Engine traceback:
  in `fmt` goal

InvalidLockfileError: You are using the `build-tools` lockfile at lockfiles/build-tools.lock with incompatible inputs.



- The inputs use interpreter constraints (`CPython<4,>=3.7`) that are not a subset of those used to generate the lockfile (`CPython==3.9.15`).- The input interpreter constraints are specified by your code, using the `[python].interpreter_constraints` option and the `interpreter_constraints` target field.
...

I haven't specified the inputs' constraints (

<4,>=3.7

) anywhere; I in fact use a fixed python version for everything. My

pants.toml

has:

[GLOBAL]
pants_version = "2.16.0rc0"

[python]
enable_resolves = true
default_resolve = "default"
interpreter_constraints = ["CPython==3.9.15"]

[python.resolves]
default = "lockfiles/default.lock"
build-tools = "lockfiles/build-tools.lock"

[black]
install_from_resolve = "build-tools"

# and the same for lots of other tools -- autoflake, isort, and so on

My tools are specified in `/pants/build_tools/`:

# /pants/build_tools/requirements.txt
autoflake==2.0.2
black==22.3.0
flake8==6.0.0
isort==5.11.5
mypy==1.1.1

# /pants/build_tools/BUILD.pants
python_requirements(resolve="build-tools")

The lockfile header mentions my actual interpreter constraint:

# /lockfiles/build-tools.lock
// This lockfile was autogenerated by Pants. To regenerate, run:
//
//    pants generate-lockfiles --resolve=build-tools
//
// --- BEGIN PANTS LOCKFILE METADATA: DO NOT EDIT OR REMOVE ---
// {
//   "version": 3,
//   "valid_for_interpreter_constraints": [
//     "CPython==3.9.15"
//   ],
//   "generated_with_requirements": [
//     "autoflake==2.0.2",
//     "black==22.3.0",
//     "flake8==6.0.0",
//     "isort==5.11.5",
//     "mypy==1.1.1"
//   ],
//   "manylinux": "manylinux2014",
//   "requirement_constraints": [],
//   "only_binary": [],
//   "no_binary": []
// }
// --- END PANTS LOCKFILE METADATA ---
...

Can anyone tell me what I'm doing wrong?

Hi! Regarding the new python.providers.experimental.pyenv backend, when I enable it, it installs 3.7 despite having

interpreter_constraints=[">=3.8,<3.9"]

. There is no docs for this, so maybe there is some configuration that needs to be done?

A demo repo for Apache Airflow users using Pants: • https://github.com/da-tubi/pants-airflow • https://airflow.apache.org/docs/apache-airflow/stable/start.html Compared to the quick start of apache airflow, the pants way is much more more more more more more quicker.

Where do resource files get packaged inside of a pex? I have my resource target explicitly added as a dependency to my pex_binary target, but I'm not seeing the resources files anywhere when I explode my pex to a venv?

In BUILD files, is there a way to take an explicit dependency on something like "uvicorn[standard]" - or would I need to individually add each dep?

has anyone tried finding a way to use pex files with Databricks? I foresee some relatively major complications arising as a result of needing to get things installed on the cluster workers, seems like if possible it would require some pretty hacky workarounds like unzipping the pex and installing the included dependencies somehow. probably just better off creating a wheel and an entrypoint but just curious if anyone has tried it and gotten anywhere

Hi, I am trying out the SCIE pants and specify pants 2.16.0rc0 in pants.toml. This file is put under a subdirectory of my repo so before I have an env var PANTS_TOML to indicate that. however I found this is not going to work. If I move pants.toml to the top level it would work.

Noticed the preliminary node/js support in 2.17.*.dev! Is there a blog post or documentation yet on how to write BUILD files for this new subsystem? If not, looking forward to the announcement 🙂.

How does pants expose a python package entrypoints to a pex? I want to make sure if I define an entry_point in a library, that I can discover it from within a pex application.

Do any of the maintainers use vscode? Trying to develop a plugin but I can't get vscode quick fix to work 🙁

I'm getting errors when trying to run the

generate-resolves

goal in

2.17.*

.

19:42:06.01 [ERROR] 1 Exception encountered:

Engine traceback:
  in `generate-lockfiles` goal

ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 2.
stdout:

stderr:
usage: pex_v2.1.130 [-h] [-V] {interpreter,lock} ...
pex_v2.1.130: error: a subcommand is required

I am working through tutorial here https://www.pantsbuild.org/docs/create-a-new-goal and (have not got to running the code) i suspect, unless there is magic sauce I am not familiar with, the example has a bug

# project_version/targets.py
from pants.engine.target import COMMON_TARGET_FIELDS, SingleSourceField, Target


class ProjectVersionTarget(Target):
    alias = "version_file"
    core_fields = (*COMMON_TARGET_FIELDS, SingleSourceField)
    help = "A project version target representing the VERSION file."

# project_version/register.py
from typing import Iterable

from pants.engine.target import Target
from project_version.target_types import ProjectVersionTarget   <<< incorrect import ?


def target_types() -> Iterable[type[Target]]:
    return [ProjectVersionTarget]

the import in the register example

from project_version.target_types import ProjectVersionTarget

should be

from project_version.targets import ProjectVersionTarget

😆 - okay so I am back at the same place I was 7 months ago. https://pantsbuild.slack.com/archives/C046T6T9U/p1663657211424529 I have a file (ephemeral in the sandbox - VERSION, generated from the vcs_version) .. i want to "read that file" and supply it to the version field on the "python_artifact" GOAL - write read_version(...) to supply the pre-computed version.

resource(
    name = "pyproject_toml",
    source = "pyproject.toml",
)

vcs_version(
    name = "version",
    generate_to = "VERSION",
    template = "{version}",
)


python_distribution(
    name = "myproject",
    dependencies = [
        ":pyproject_toml",
    ],
    generate_setup = True,
    provides = python_artifact(
        name = "myproject",
        version = read_version(
            dependencies = [
                ":vcs_version"
            ]
        )
    ),
    ...
)

Our integration tests require a Kubernetes cluster. To ensure we can run multiple CI pipelines in parallel (e.g. for different PRs) we have a pool of clusters available. When a CI pipeline starts, it selects one cluster from the pool and executes the test on it. Currently we do this by setting an env var that is picked up by the setup code for the tests. When using pants to run our integration tests we can pipe this env var through without problems. But since the value of the env var can change between different runs we can not fully benefit from the test caching of pants. Is there a better way to provide the cluster information to the test suite without invalidating the cache?

What would be required to look at resolving https://github.com/pantsbuild/pants/issues/16565? As it stands this is forcing me to use pyenv in my CI rather than

actions/setup-python

, and having to run pyenv is contributing to frustratingly long build times. It's possible I could find some time to work on a fix but I'm (a) new to the pants codebase and (b) not sure how to approach solving this.

I’m trying to debug very slow sandbox requirements.pex building. Is there a flag to see the output of what it’s trying to do, as it’s doing it? E.g. what packages are being fetched, if its using a source package and therefore compiling from scratch, etc?

Hi, I'm facing a problem with scripts such as

uvicorn

when using

./pants export

I'd like to be able to sometimes run them without actually invoking pants. The reason is we've got some code in the repo that we do not package and only run during development or as pocs, and it is often easier and faster to just run it this way. The problem I'm facing is that the scripts ignore

PYTHONPATH

variable so I'm unable to import my first-party code. I compared the scripts exported using pants and the ones created with

pip install

and the only difference is in the shebang - the pants ones include

-sE

flags. This explains why they ignore

PYTHONPATH

. I assume this is by design. What's the rationale behind it? Is there a way to get rid of the

-E

flag?

Hi, what is the recommended way to build platform wheels (as opposed to pure python wheels)?

In a monorepo, is it possible to have different .pylintc files for each project? I tried to add files to the root of the projects, but it doesn't seem to work.

I'm writing a plugin that will generate some artifacts that can be fed to a downstream deployment process. Configurations for these artifacts can either be specified in a custom target defined in the plugin, in a file referenced by the custom target, or some fields can have global defaults set in the subsystem. Is there a request I can use to get pants to run an arbitrary Python script in parallel for each

FieldSet

? Similar to

Process

except pants can run it in its own Python environment.

Hey team. Any support for running Go analyzers as part of

check

or

compile

like Bazel does via nogo?

Hi community, Thanks for helping us all the time! I need a help and ask if you have any good idea. Is there any good idea to use syrupy by pants? In our case, we use syrupy that manages expected output of test. To update snapshot, we run

pytest --snapshot-update

. But pants doesn’t update snapshot.

Is there a way to have a base docker image, and build other docker images from that base? So, build a base python image with certain installations, then having the other docker images reference that image in the FROM command?

is it possible to make pants use some other name for build files instead of BUILD? I am trying to use nuitka with pants on my mac, and nuitka tries to create a “build” directory, and it’s not allowed to do that because my mac considers “build” and “BUILD” to be the same path

So I've finally got to the point in my plugin development where I'm getting exploding rule graph errors. Is it legal to use a field set as a hydration request? I thought it seemed convenient to have my field set inherit from my hydration request because it requires all of the same fields. My hydration rule also requires the subsystem and an

Address

. Do I need to define a rule telling pants how to get the subsystem and address out of my field set? I was under the impression it would already know how to do that.

Sooo I have a weird problem with our pants migration... Currently I am using libcamera and unfortunately all of the bindings and shared libraries need to be built from scratch (Currently working with the maintainers to figure out a release process but that is awhile off as FOSS). At the end of the day, meson+ninja does the build of C+ and python bindings.. but it only puts the python bindings into the local host site-packages. In our current world, we are using Make and we do this nasty blob to copy things from site-packages into a virtual environment..

$(SITE_PACKAGES)/libcamera/__init__.py: pyenv-virtualenv requirements-libcamera-deps /usr/local/lib/x86_64-linux-gnu/libcamera.so
	rm -rf .lc_build_temp
	mkdir -p .lc_build_temp
	cd .lc_build_temp \
		&& git clone <https://github.com/Exclosure/libcamera.git> \
		&& cd libcamera \
		&& git checkout v0.0.4 \
		&& PKG_CONFIG_PATH=$(PKG_CONFIG_PATH) meson setup build -D pycamera=enabled \
		&& ninja -C build \
		&& echo "Copying libcamera to venv" \
		&& cp -Lr ./build/src/py/libcamera $(SITE_PACKAGES)/libcamera
	rm -rf .lc_build_temp

Gross I know, but.. yeah. So question: Is there a way to have pants look on the local host for packages else where? Can we use third party dependencies local requirements in a creative way? Unfortunately, the built package is not a wheel or sdist.

This error just started popping up any time I try to run

pants package ...

, it seems to get thrown when pants is looking for an interpreter. I tried the troubleshooting tips it suggested to no avail. Any ideas?

Traceback (most recent call last):
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/bin/pants", line 8, in <module>
    sys.exit(main())
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 123, in main
    PantsLoader.main()
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 110, in main
    cls.run_default_entrypoint()
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 92, in run_default_entrypoint
    exit_code = runner.run(start_time)
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/bin/pants_runner.py", line 89, in run
    return remote_runner.run(start_time)
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/bin/remote_pants_runner.py", line 123, in run
    return self._connect_and_execute(pantsd_handle, executor, start_time)
  File "/Users/zach/Library/Caches/nce/142382d6e3ac0960e7d4f81b9df9de9559b7ac698db13d20cbd0e4aa57f68c1d/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/bin/remote_pants_runner.py", line 161, in _connect_and_execute
    return PyNailgunClient(port, executor).execute(command, args, modified_env)

native_engine.PantsdClientException: The pantsd process was killed during the run.

If this was not intentionally done by you, Pants may have been killed by the operating system due to memory overconsumption (i.e. OOM-killed). If you keep seeing this error message, try the troubleshooting steps below. If none of those help, please consider filing a GitHub issue or reaching out on Slack so that we can investigate the possible memory overconsumption (<https://www.pantsbuild.org/docs/getting-help>).
 - Exit other applications, including applications running in the background.
 - Set the global option `--pantsd-max-memory-usage` to reduce Pantsd's memory consumption by retaining less in its in-memory cache (run `./pants help-advanced global`).
 - Disable pantsd with the global option `--no-pantsd` to avoid persisting memory across Pants runs, although you will miss out on additional caching.

Hi, I am trying to migrate from this project structure:

project/
--A/
----lib_A/
------BUILD
----dist_A/
------BUILD
--shared_dist/
----BUILD

where the sources in

project/A/lib_A

have a dependency on

project/shared_dist

and

project/A/dist_A

. The directory

project/A/dist_A

builds a package that has 3rd party dependencies. (FYI, when this package runs, it loads an artifact that was produced by

lib_A

- this is why the directories are grouped together in

project/A

.)

shared_dist

also builds a package that has 3rd party dependencies. We are trying to reorganize the structure to the following:

project/
--A/
----dist_A/
----lib_A/
----BUILD
--shared_dist/
----source.py
----BUILD

where

project/A/dist_A

now builds a package that contains the sources in

dist_A

, the sources in

lib_A

and the dependencies from

shared_dist

. My first attempt of running

./pants package …

resulted in the error

NoOwnerError: No python_distribution target found to own project/A/shared/path/to/file.py:../lib@resolve=my_resolve. Note that the owner must be in or above the owned target's directory, and must depend on it (directly or indirectly).

It seems that the package is unable to pull in the resources in the

shared_dist

package directory. The link to the documentation on python-distributions indicates this is possible, so I’m not sure where my mistake lies. Does anything jump out at you or do you need more information?

Hi all! I'm trying to update my Poetry dependencies to include the

sagemaker

Python SDK, and some of the runs for a

pants check ::

(and other goals like package) are going on over 3 hours now, compared to if I remove that dependency it's 5 seconds. How can I best debug this and/or anyone have some ideas?