Following up on my questions above, as i progress in the exploration - 1. Our project runs on both linux and macos. The requirements lock is a bit different on those systems (we use platform=='Darwin' inside requirements.txt for some packages). What is the right way to treat different lock files with pants? 2. I saw that currently there is no option tto invoke cross platform remote execution. E.g. can't call linux rbe from macos. Does anyone know when such support is expected? 3. Is there support for building for Windows as the target system?

We're at 2499 Github stars right now. Who wants to do the honors of being 2500?

I'm minding my own business editing a pants.toml in PyCharm, and suddenly I get context-sensitive autocomplete for every scope and option, without having to do anything at all. Of course I quickly realized this is due to https://github.com/pantsbuild/pants/issues/17895. Huge hat-tip to @fresh-cat-90827 for this!!

I'm trying to get an idea of how long each step takes while building an image (doesn't necessarily have to be exact, even just me seeing it on the screen is enough), but even with

-ldebug

the output is buffered then dumped immediately, is there a way to do that?

I'm trying to use https://pypi.org/project/pytest-rabbitmq/ in a unit test. It sets up a fixture that starts/stops a rabbitmq server for use in a test. I got this working outside of pants with

pytest

so I know my installation is good. It requires a

rabbitmq-server

and

erlang

installation on the system which are both available under my

PATH

. Pants seems to be picking things up fine because the server is attempted to start, but fails the boot process. I think sandboxing is the most obvious thing at play here. Any ideas for environment variable differences or similar to look at? When I preserve sandboxes, the generated

__run.sh

executes without error.

Hello, we have a use case where we want to package some files into a zip file using an

archive

target deterministically (in 🧵).

hi - i just removed my

./pants

script from the repo, and added

pantsup.sh

- im getting a flake - seemingly only in CI and not local, although that could just be because im testing more in CI - typical error is:

✓ mypy succeeded.
FATAL: exception not rethrown
FATAL: exception not rethrown
/home/runner/work/_temp/f3725c23-b66e-48a4-b13a-9692347a0610.sh: line 1:  1684 Aborted                 (core dumped) ./scripts/pants check ::
Error: Process completed with exit code 134.

i can open a ticket but i wanted to rule out that im doing something incorrect first

Is there a way to pass parameters to docker build? I tried adding a shell script as the docker executable, but that doesn't seem to work, even if it's just doing

docker "$@"

Hello, maybe I’ve missed it in the docs - is there a way to manage pip constraints files in python dependencies? (more info here -> https://pip.pypa.io/en/stable/user_guide/#constraints-files) Usually, that helps a lot for dependency resolution, therefore it should make lockfiles generation faster 🙂

Hello! I am using pants export to setup my IDE. It does work, i.e. in /dist/export/python/virtualenvs/default/3.8.15/lib/python3.8/site-packages I can find everything I need. Is there any option to force the name of the folder 3.8.15 just to be 3.8?

I'm evaluating Pants for a repository and I'm having trouble understanding build and test works for other versions of python. Currently my CI uses python 3.8 but I packages builded with python 3.9. Everytime I try to export the venv I get that error message:

image.png

Hello folks! We are currently trying to accelerate our pipeline with the remote cache. We are trying to register on Toolchain but we can't get the email when we register. We also tried to run on-prem version of Buildbarn or Buildfarm with this in my

pants.toml

remote_cache_read = true
remote_cache_write = true
remote_store_address = "<grpc://cache.local:8980>"
remote_instance_name = "worker"

My on-prem REAPI server are working correctly and exposed correctly (Tested with Bazel example) and when I try the

remote_execution

with Pants, I get errors about the Environment missing Python (which is normal at this state) but we don't want remote execution, only the caching. When running any command

lint, fmt, test, package, publish

I don't see any hit on my server or logs in Pants that tell its trying to do something with the remote cache, Am I missing something here ? I am following this https://www.pantsbuild.org/v2.15/docs/remote-caching

We made our Pants CI pipeline mandatory at my client, and so developers start having concerns that our Pants pipeline is slower than the regular CI pipeline. The crux is that our regular CI spawns a test job per library of our monorepo being modified, whereas the Pants pipeline simply runs

./pants --changed-since=origin/main --changed-dependees=transitive test

. And because GitHub default runners have pretty low performance, the Pants CI is on the average 4 to 5 times slower, because it uses a single machine. I wonder where I should be headed to speed this up. For the moment we don't cache Pants' state and use

--changed-since=origin/main

, because downloading the cache was quite slow. Ways forward that I see: 1. Use persistent runners, keep Pants' state 2. Use remote executors 3. Use a remote cache Any experience to share on what would yield the biggest speedup?

Hi guys, facing the same issue mentioned here. The docker environment doesn’t actually spinup docker container and runs on local.

~/bin/pants package clients/prefect:prefect_client
00:16:55.48 [INFO] Completed: Building clients.prefect/prefect_client@environment=local_docker.pex with 2 requirements: prefect[aws,postgres,redis,templates,viz]<0.16.0,>=0.15.6, redis-py-cluster<3.0.0,>=2
00:16:55.63 [INFO] Completed: Building clients.prefect/prefect_client@environment=local.pex with 2 requirements: prefect[aws,postgres,redis,templates,viz]<0.16.0,>=0.15.6, redis-py-cluster<3.0.0,>=2
00:16:55.69 [INFO] Wrote dist/clients.prefect/prefect_client@environment=local_docker.pex
00:16:55.69 [INFO] Wrote dist/clients.prefect/prefect_client@environment=local.pex

It’s using the right environment, but doesn’t use docker at all (details in 🧵)

Posting this in #general because it is a widespread issue. Yesterday GitHub changed their compression for git archives. Meaning URLs like:

http_source(
        filename = "pybind11",
        len = 569736,
        sha256 = "8103bf774f02ff7a063c017eb1adac51c17af9c2c08363830618ad648f09ad1f",
        url = "<https://github.com/pybind/pybind11/archive/refs/tags/v2.4.3.tar.gz>",
    ),

Started failing due to the sha/len being different. And then today, they decided to revert the change. So just FYI to those of y'all fielding off errors from Pants talking about unexpected length/SHA on GitHub tarball URLs.

# pants.toml
[GLOBAL]
pants_version = "2.15.0rc2"
pantsd = false
print_stacktrace = true
pythonpath = ["%(buildroot)s/build-support/plugins"]
colors = true
build_patterns = ["PANTS", "PANTS.root"]
build_file_prelude_globs = ["build-support/macros.py"]

backend_packages = [
  "pants.backend.python.lint.pyupgrade",
  "pants.backend.python.lint.autoflake",
  "pants.backend.python.lint.docformatter",
  "pants.backend.python.lint.isort",
  "pants.backend.python.lint.black",
  "pants.backend.python.lint.flake8",
  "pants.backend.python.lint.pylint",
  "pants.backend.python.typecheck.mypy",
  "pants.backend.python",
  "pants.backend.shell.lint.shellcheck",
  "pants.backend.shell.lint.shfmt",
  "pants.backend.shell",
  "pants.backend.awslambda.python",
  "pants.backend.plugin_development", # for the pants_requirements macro
  "pants.backend.docker",
  "pants.backend.docker.lint.hadolint",
]

[python]
interpreter_constraints = [">=3.8,<3.10"]
interpreter_versions_universe = ["3.8", "3.9"]

[python-bootstrap]
search_path = ["<PATH>"]
names = ["python3.9", "python3.8", "python3", "python"]

....

[environments-preview.names]
local_docker = "//:local_docker"
local = "//:local"

And

PANTS.root

is :

...
docker_environment(
    name="local_docker",
    image="python:3.9.13-slim-buster",
    fallback_environment="local",
    docker_executable_search_paths=["<PATH>"]
)

local_environment(
  name="local",
)

# __defaults__(all=dict(environment="docker"))
...

clients/prefect/PANTS

is :

python_sources()
pex_binary(
    name="prefect_client",
    entry_point="prefect_client.py",
    environment=parametrize("local_docker", "local")
)

Hi all, I’m having some issues with running pants in CI that I can’t reproduce locally. I’ve managed to reduce the issue down to something simpler, but I’m unsure how to proceed I have a file called

lib/platform/tests/test_advice_models.py

which imports from

lib.business_rules

import lib.business_rules  # noqa


def test_thing():
    assert 1 == 1

Running this locally, everything works completely fine ✅

$ ./pants test --force lib/platform/tests/test_advice_models.py
15:51:25.09 [INFO] Completed: Run Pytest - lib/platform/tests/test_advice_models.py - succeeded.
Partition: platform

✓ lib/platform/tests/test_advice_models.py succeeded in 0.85s.

But running the same test in CI gives an error ❌

==================================== ERRORS ====================================
__________ ERROR collecting lib/platform/tests/test_advice_models.py ___________
ImportError while importing test module '/tmp/pants-sandbox-5gjBP7/lib/platform/tests/test_advice_models.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
lib/platform/tests/test_advice_models.py:1: in <module>
    import lib.business_rules  # noqa
E   ModuleNotFoundError: No module named 'lib.business_rules'

My GH action is invoking the exact test file (the same way I would run locally)

jobs:
  build_and_test_python:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.9]
    steps:
    - uses: actions/checkout@v3
    - uses: pantsbuild/actions/init-pants@v2
    - name: Test
      run: |
        ./pants test lib/platform/tests/test_advice_models.py
      if: always()  # We want the log even on failures.

Checking the output of

./pants dependencies --transitive lib/platform/tests/test_advice_models.py

Includes things in

lib.business_rules

so unsure why CI is unable to find this module

lib/business_rules/__init__.py
lib/business_rules/engine.py
lib/business_rules/operators.py

It’s a long shot (apologies), but any idea how I could begin to debug what’s going on here? (I’m on pants 2.15.0rc2 )

im trying to exclude some files/directories/anything from

docformatter

- does anyone have any ideas how this could be acheived?

If a

python_test

target depends a

python_distribution

target (a packagable target) via the

depenedencies

field instead of the

runtime_package_dependencies

field[1], will the

python_test

transitively depend on everything the

python_distribution

depends on (like the deps inferred from entry_points)? [1] https://www.pantsbuild.org/docs/python-test-goal#testing-your-packaging-pipeline I want to depend (well, actually infer a dep from within a plugin) on the underlying code without actually generating the wheel.

Hi all, My college @flaky-battery-655 and I wanted to reach out to the community regarding our use case at INGU for pants build to see if there are any suggestion around how we are currently and planning on using it. We are moving away from multiple python repositories into a monorepo with multiple projects. The planned final structure is: - monorepo - apps - example-gui-tool.py - assets - src - python-api - math - models - plots - utils - tests Many of the tools in

apps

are used by non-developers in parts of the company that will depend on the algorithms written in

src

. This will also allow a more formal development process into

src

to be treated similar as numpy and pandas for internal development. Everything up to this point had been built exclusively in a windows environment. Setting up to use pants to handle our dependencies, testing, and distribution we have started working in WSL using ubuntu 22.04. My understanding that in addition to being able to run pants this should help us remove any incurred dependencies on windows enabling easier distribution to other operating systems or servers. The questions we have at the moment are: • What is the best way to distribute standalone python apps to internal workers on windows machines? ◦ We have been reviewing the docs but the support seems to centered towards pex (https://www.pantsbuild.org/docs/python-distributions) • Some members of the development team work in pycharm while others us vs code. Our current plan to to leave most development flows unaltered in the previous windows environments with pycharm and handle testing/distribution separately in ubuntu or using the WSL extension for vs code. Does this raise any red flags? In addition, we would very much appreciate any feedback regarding our proposed structure and plan this far! Let me know if there are any other details that would help in understanding our questions or if you have any questions for us!

I'm the process of migrating some unit test for a

pyspark

application over to our pants repo. Previously, I was running these test inside a docker image built with the databricks runtime as their base image. Java exists on said image as it is a dependency of spark and thus databricks. Now the following fixture no longer works as Java isn't install on my machine:

@pytest.fixture(scope="session")
def spark_session() -> SparkSession:
    return (
        SparkSession.builder.master("local")
        .appName("spark session example")
        .getOrCreate()
    )

Is there a way for pants to manage the setup/installation of java for this test suite?

Hey all, I’m having a hard time getting the file system watching capabilities of pants working. If I run:

./pants run src/python/app/run.py

Then make changes to

run.py

or any of its transitive dependencies, I would expect pants to restart the run command. What am I missing here?

Is the file owned by a

pex_binary

target with this field set?: https://www.pantsbuild.org/docs/reference-pex_binary#coderestartablecode

Yes. I’ve tried pointing pants at my pex binary target that builds the app as well, but go the same results. Let me try again.

Are you on Windows WSL?

If so, Pants must be used only against a repo fully housed in WSL vm partitions not /mnt/c etc.

I was! But I’ve switched back to a mac to see if that was the issue. That might have very well been the problem!