Does pants do something special to render terminal output with color? I'm hooking unit test execution into our change submission process and tests are running happily as I'd expect, but the stdout/stderr I'm able to print to the screen is missing its color. We didn't have this problem with our original hook that calls pytest.

Still redirects through

<http://v1.pantsbuild.org/setup/pants>

Currently the docs are very geared toward those who setup, configure, and administer Pants. We've heard from many of you that part of your adoption process has been to produce simplified custom internal documentation for your end-users to refer to. We'd like to create some end-user oriented docs similar to that, to make that step easier for future adopters to accomplish. Something they can refer people to generically, and can easily adapt to incorporate internal details. (a) What is a good MVP for that? (b) Please do share your end-user docs, if you'd be comfortable, to help us identify what to cover. We want to learn from your deeper knowledge of what those end-users need.

Is there any way to refer to a target relative to the current path (and just going further in the tree, not walking back up)? If the target I'm interested in doesn't live in the current directory, it seems like I'm forced to specify the target path relative the repo root.

Weird one: I've got a test. It has a dependency that I've specified and is working as expected (i.e. I see it when I do

./pants dependencies path/to/my_test.py

). When I do

./pants dependencies path/to/depended_on_thing

, I see the correct dependencies I'd expect there. However, when I do

./pants dependencies --transitive path/to/my_test.py

, I don't see the correct dependency of

depended_on_thing

. Apologies for the genericized version of this, but any ideas?

Is there any way to get a summary output style like bazel has?

INFO: Elapsed time: 715.011s, Critical Path: 105.07s
INFO: 624 processes: 4 remote cache hit, 670 linux-sandbox.
INFO: Build Event Protocol files produced successfully.
INFO: Build completed, 1 test FAILED, 624 total actions

It's way nicer than seeing the result of every test as pants does it (it seems like

--test-ouput

just controls details of each executed test and not limiting the information dump to the terminal).

How does Pants decide how many jobs to run for parallel invocations? I'm running

./pants lint ::

with the

flake8

and

pylint

backends enabled, and it seemed to have run flake8 massively parallel, and pylint in just one job. Is this an intentional decision due to bugs in pylint?

I'm going to preface this by acknowledging that it's a weird question: Is there a way to specify different sets of

pytest

configuration declaratively (or even non-declaratively!) for different subsets of tests? Something like having different extra-requirements (https://www.pantsbuild.org/docs/reference-pytest#section-extra-requirements) for unit tests vs. integration tests?

Re-upping this for additional feedback. Suggestions much appreciated, folks! https://pantsbuild.slack.com/archives/C046T6T9U/p1643222888752119

I feel like I saw this in the docs somewhere, but is there a way, kind of like a unit test, to make sure that

./pants tailor

doesn't have any changes it would want to apply?

./pants run

seems to be interfering with python signal handling in a way that is causing context managers to not get cleaned up -- wondering if there is a workaround for this. as an example, if i take the following simple script:

from typing import Iterator
from contextlib import contextmanager
import time

@contextmanager
def test_sync_ctx() -> Iterator[None]:
    print("[test_sync_ctx] creating...")
    time.sleep(1.2)
    try:
        yield None
    finally:
        print("[test_sync_ctx] closing...")
        time.sleep(1.2)
        print("[test_sync_ctx] closed")


def main() -> None:
    with test_sync_ctx():
        print("inside sync context. sleeping...")
        time.sleep(5)
        print("slept.")


if __name__ == "__main__":
    main()

Running directly with python, i get the following output:

[test_sync_ctx] creating...
inside sync context. sleeping...
^C[test_sync_ctx] closing...
[test_sync_ctx] closed
Traceback (most recent call last):
  File "test_ctx.py", line 26, in <module>
    main()
  File "test_ctx.py", line 21, in main
    time.sleep(5)
KeyboardInterrupt

But using pants, no cleanup is ever run. I get the following output:

[test_sync_ctx] creating...
inside sync context. sleeping...
^CInterrupted by user.

Any thoughts on what, if any, would be the equivalent of a hermetic python in bazel for pants? We'd really like to avoid using anything on system as much as possible since it's caused us issues in the past.

Hmm, running into some issues with Pants 2.10.0.dev2. See thread...

I'm creating a

cookiecutter

(a templating tool - https://cookiecutter.readthedocs.io/en/1.7.2/) project that creates a Pants project, and is itself a Pants project. Everything works great, but I want to have Pants ignore any

BUILD

files within the body of the template itself (because those

BUILD

files may themselves be templates, and thus not valid files for Pants to process as-is). I can do this with the following stanza in my top-level

pants.toml

file:

build_ignore = [
    # Don't want the BUILD files in the template to get treated as
    # though they applied to this project.
    "\\{\\{ cookiecutter.__repo_name \\}\\}"
]

(Yes, that directory name --

{{ cookiecutter.__repo_name }}

-- is a Jinja templating directive; that's just how

cookiecutter

works.) That complicated escaping is the best thing I've been able to come across that is accepted and lets me actually run Pants. However, when I run Pants with this configuration, I get a slew of repeated messages like this:

12:50:50.87 [WARN] <string>:1: DeprecationWarning: invalid escape sequence \{

The

pants.toml

is the only place in the project that has this character sequence. I'm not exactly sure what escape sequence would work here (again, nothing else I've thought of seems to be accepted). The

DeprecationWarning

also makes me concerned that the next release of Pants will no longer work with this repository. Anybody else seen anything like this or have any suggestions? Thanks 🙇

How would I control

test

to run serially for some specially marked tests? I don't see anything in https://www.pantsbuild.org/docs/reference-test or https://www.pantsbuild.org/docs/reference-pytest for an option on that. We have integration tests that have to run against shared hardware and running in parallel would cause lots of issues.

Hey Pants Team, we're seeing an issue after an engineer upgraded their laptop OS to Monterey. They are unable to use the

--changed-since

subsystem and the error message is saying the git dir cannot be found. All other pants goals work without using the subsystem so curious why this is occurring.

I'm working on a new section of our codebase and I'm seeing test targets take a really long time to complete. When I run them individually, they're taking around 5 seconds for pants to do its thing and running concurrently I'm seeing them take upwards of 15 to 20 seconds. The actual tests that run are really simple and only take 0.5 second at most (not accounting for some initial setup that runs in conftest that take around 1.5 seconds). Any ideas why I might be seeing such a large discrepancy?

Hi all, I'm testing out pants on a repo at work, but my workstation doesn't have access to the internet. I'm blocked right now because pants is trying to download pex from github.com/pantsbuild/pex/releases/download and failing. We have an internal pypi proxy, so I can pip install pex, but I can't use curl, etc to download arbitrary files over http(s). Is there a way to force pants to use pex found on

PATH

? Or can I manually download the pex release (we use an airgapped secure browser w/ download and virus scan capabilities) and place it in a known location for pants to find? We also have a docker proxy setup, so I also looked on dockerhub for a container that would hold all the required deps, but the only container there was 6 years old. For users like myself without internet access it'd be great to bundle everything up in an easy to use package. Thanks!

Hi all, I'm trying to set-up the following integration testing environment using Pants and I'm getting stuck. We are currently using

docker-compose

to create environments for running integration tests. We might want to move to using Kubernetes pods in future. In both cases we are running the tests from a container which should be created by the test framework. This is different from the way pants runs tests - they are executed in the same environment as pants. Ideally I would like

./pants test ./test/my-integration-test.py

to do the following: 1. Set up a docker-compose test environment. The necessary data for it should be collected from the target. 2. Build the necessary pex files for the test 3. Copy the pex files to a container in the environment 4. Execute the test from the environment and collect the results 5. Tear-down the test envrionment Do you think that something like this is possible to achieve? I'm open to writing plugins as long as I don't need to rewrite the whole pytest rule to get this thing to work.

hey team 🙂 I am working on a blog post that should help Pants users to decide on the strategy of choosing the Python interpreter for their project. This is a massive topic, but I hope I was able to highlight at least some of the key things to keep in mind and shed light on pros and cons of what I think would be sensible strategies. Comments are very welcome! https://docs.google.com/document/d/1IQk3pM3YKLypwvw0Y8vMq8jRAJ9J0n-0xZ-GhbuFPE4/edit?usp=sharing @busy-vase-39202

Hey all, I'm trying to gauge the best way to contend with dependencies in our monorepo. Today we don't have many teams on PantsV2 yet, but we envision a day where we'll have hundreds of projects, and hundreds of teams (who don't communicate much per se) in our monorepo using pantsv2. We're already hitting a wall with 2 teams; migrating our second team onto pantsv2 involved them registering a requirements.txt file with python requirements, and it caused team 1s project to start failing because of dependency collisions. We know we're probably not handling dependencies optimally for pants, but we're unsure exactly what optimally is. Given where things stand with 2.9, it seems like we should be planning on explicitly registering every dependency manually in build files as opposed to registering with python_requirements(). @hundreds-father-404 was showing me some work he was doing in 2.10 to make things a little more manageable, but I don't know if there's anything else we're missing

Probably a stupid question, but is there a good reliable way to run pants commands with nice? I guess once pantsd is started you have to renice it, but you also never know when pantsd will need to be restarted, so you'd have to nice the invocation of

./pants

too? Like this:

renice 19 $(cat .pids/**/pid)
nice ./pants ...

Does this make sense or am I thinking about this all wrong?

Hey folks, I could use some help trying to track down an error with pants trying to install a python package. The package is https://pypi.org/project/pyranges/, which only distributes as source code. My pex_binary target declares a dependency on it, but when I try to run it, this is what I'm getting:

ProcessExecutionFailure: Process 'Building src.[...]/exe.pex with 10 requirements: boto3==1.20.21, cryptography==3.4.8, numpy==1.21.4, overrides==6.1.0, pandas==1.3.5, pyranges==0.0.111, scipy==1.7.3, setuptools==60.7.1, toml==0.10.2, types-toml==0.10.1' failed with exit code 1.
stdout:

stderr:
ERROR: Could not find a version that satisfies the requirement pyranges==0.0.111
ERROR: No matching distribution found for pyranges==0.0.111
ERROR: Could not find a version that satisfies the requirement pyranges==0.0.111
ERROR: No matching distribution found for pyranges==0.0.111
pid 34310 -> /Users/jwarwick/.cache/pants/named_caches/pex_root/venvs/7f518a293766277276f42aa896ce105b056e5634/c31a5b9c4ba316e0b508d32fa17316ce30a6f329/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/jwarwick/.cache/pants/named_caches/pex_root --log /private/var/folders/3k/1rb9kxyn35z51bc7ny7z1xgw0000gn/T/process-executionC9Rqy4/.tmp/tmpu542abw0/pip.log download --dest /private/var/folders/3k/1rb9kxyn35z51bc7ny7z1xgw0000gn/T/process-executionC9Rqy4/.tmp/tmph7y1egmo/manylinux2014_aarch64-cp-39-cp39 --platform manylinux2014_aarch64 --implementation cp --python-version 39 --abi cp39 --only-binary :all: boto3==1.20.21 cryptography==3.4.8 numpy==1.21.4 overrides==6.1.0 pandas==1.3.5 pyranges==0.0.111 scipy==1.7.3 setuptools==60.7.1 toml==0.10.2 types-toml==0.10.1 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None

My guess is that this has to do with

--only-binary :all:

. When I try to install this package with pip in a fresh virtualenv, there is no error. Can anyone point me in the right direction?

Hi Pants! I have some questions in the pants / pex caching weeds. We’re currently trying to reduce the amount of added CI / CD time incurred by adding or updating 3rdparty dependencies in our monorepo. I’ve been following https://github.com/pantsbuild/pants/issues/14127 and was wondering if there are any workarounds that resemble incremental 3rdparty dependency resolution. Does Pants re-install all 3rdparty dependencies from indexes if it can’t find the full resolve in its cache? I noticed that

pex

also caches

installed_wheels

in

~/.cache/pants/named_caches/pex_root

. Is Pants (through

pex

) able to consult that cache when re-resolving dependencies? In other words, does it still re-install packages from indexes even if the exact package / version is in that

pex

named cache? We currently save / restore the three Pants cache directories in our build pipelines and bust these caches whenever our 3rdparty lockfile changes. I’m wondering if persisting

~/.cache/pants/named_caches/

even in the case of a 3rdparty lockfile change would allow the build to not have to re-download packages from indexes, therefore saving time. Do these questions / this line of thought make sense? Thanks in advance 🙏

my build system of choice is CMake. I see that others are working on Pants with C++. I wonder what the difference is between Nx and Pants. Neither support CMake directly but both claim to be technology agnostic. I am hoping to avoid Nx because it uses Javascript. I'm more comfortable with Python.

CMake offers JSON files called presets. They're used to configure builds but are meant to be used on a per project basis. Presets don't support source roots discovery and only supports CMake. I wonder if Pants could replace presets or if a combination of Pants + presets makes sense.

How do I run tests sequentially considering the same

python_tests

?

Random comment. A good starting point for getting a feel for the Pantsbuild system as a new user is the “Key concepts” chapter under “Using Pants”: https://www.pantsbuild.org/docs/goals 😉

I have some existing code that reads in some yamls files and compiles some python modules and places them in the repo. I'm wondering if there's any way in a Pants paradigm to keep this kind of functionality. It sounds like plugin territory since it's exactly like compiling proto files into

pb2.py

modules. I think I already know the answer, so this is mostly just 🎣 .

When I'm running the

tailor

goal, is there some machinery that I can hook into to influence what targets get autogenerated into the BUILD files? For instance "For every directory with a

__main__.py

add a `python_distribution`"