There is: https://www.pantsbuild.org/docs/reference-python-native-code and https://www.pantsbuild.org/docs/reference-subprocess-environment

Excellent. Thanks!

I had a question about starting to use remote caching (e.g. https://www.toolchain.com). I like the idea and we'd like to do it BUT we do something odd with our dependencies. Per a previous conversation on this Slack channel we're using PyTorch and sometimes need the GPU version of it and sometime the CPU version. Annoyingly, the CPU and GPU versions are the same pip version and the only difference between them is which

--extra-index-url

you pass to

pip

. To make things realitvely easy we control this with an env variable and a wrapper script: if the variable is set we set the

PANTS_PYTHON_REPOS_INDEXES

and

PANTS_PYTHON_RESOLVES

environment variables to pull the right dependencies. So.... given our funky setup, can we still use remote caching or is it going to be unaware of those variables and thus deliver the wrong dependencies (e.g. if I build with CPU and my collegue sets those env vars and thus expects a GPU build will they end up pulling the CPU dependencies from the cache)?

Oh hey! I have a novel way of solving the torch issue, FWIW. Was gonna blog about it in the next months or so. DM me if youre interested and I can give you the cliffs notes

So.... given our funky setup, can we still use remote caching or is it going to be unaware of those variables and thus deliver the wrong dependencies (e.g. if I build with CPU and my collegue sets those env vars and thus expects a GPU build will they end up pulling the CPU dependencies from the cache)?

Pants is aware of all environment variables fed to processes, and will do the right thing here.

Been awhile since I posted, but I just needed to drop a note saying, how much I appreciate the time

pants

saves me. I just wrote a simple script for munging some JSON, dropped it into the

scripts

folder, ran

pants tailor scripts/

, and now I can

pants run scripts/doesstuff.py -- tomy.json

on any machine I have the repo checked out without having to build a venv for my script. I wont mention I get

fmt

and all the other good practice stuff for free. 🦜👖 🎉

Thanks!

Happy to help with onboarding to Toolchain! And we have a Slack channel for support which I can send you an invite for.

Cool, welcome! What kind of help would you be most interested in? I'm sure we can connect you with something interesting.

Thanks! We definitely want to move in that direction eventually but I think we're going to back-burner for now. hopefully return to it in a few weeks.

It’s very easy, just a web signup and a config change.

When you’re ready for it

Are the arrow'd lines below due to race conditions? Running

./pants fix ::

on 2.15.0a0

20:57:36.78 [INFO] Completed: Format with shfmt - shfmt made no changes.
--> 20:57:38.46 [INFO] Canceled: Building autoflake.pex from autoflake_default.lock
20:57:39.75 [INFO] Completed: Building clang_format.pex from clang-format_default.lock
20:57:40.09 [INFO] Completed: Building autoflake.pex from autoflake_default.lock
20:57:40.44 [INFO] Completed: Fix with Autoflake - autoflake made no changes.
--> 20:57:40.46 [INFO] Canceled: Building pyupgrade.pex from pyupgrade_default.lock
20:57:40.52 [INFO] Completed: Fix with Autoflake - autoflake made no changes.
20:57:40.55 [INFO] Completed: Fix with Autoflake - autoflake made no changes.
--> 20:57:40.57 [INFO] Canceled: Building pyupgrade.pex from pyupgrade_default.lock
20:57:40.70 [INFO] Completed: pants.backend.cc.lint.clangformat.rules.clangformat_fmt - clang-format made no changes.
20:57:41.87 [INFO] Completed: Building pyupgrade.pex from pyupgrade_default.lock
20:57:42.22 [INFO] Completed: Fix with pyupgrade - pyupgrade made no changes.
20:57:42.27 [INFO] Completed: Fix with pyupgrade - pyupgrade made no changes.
20:57:42.28 [INFO] Completed: Fix with pyupgrade - pyupgrade made no changes.
20:57:45.13 [INFO] Completed: Building black.pex from black_default.lock
20:57:47.60 [INFO] Completed: Format with Black - black made no changes.
20:57:47.69 [INFO] Completed: Format with Black - black made no changes.
20:57:47.71 [INFO] Completed: Format with Black - black made no changes.
--> 20:57:47.73 [INFO] Canceled: Building isort.pex from isort_default.lock
20:57:49.05 [INFO] Completed: Building isort.pex from isort_default.lock
20:57:49.50 [INFO] Completed: Format with isort - isort made no changes.
20:57:49.56 [INFO] Completed: Format with isort - isort made no changes.
20:57:49.57 [INFO] Completed: Format with isort - isort made no changes.

Am about to embark on adding some rust development into our pipeline. is there a) a roadmap for the rust backend support b) irrespective of a), what is the best way to "generate" artefacts from shell and have them available as "dependencies" in the pants ecosystem ? (Rust is one language, we have another in the embedded space, proprietary)

Hello

pants

developers 👋, I would like to get rid of the following WARN messsages when I run tests:

The target libs/image_processing/kaiko/image_processing/mask/statistics.py:../../../kaiko-image-processing imports `numpy`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['libs/geometry:kaiko-geometry-poetry-reqs#numpy', 'libs/geometry:kaiko-geometry-reqs#numpy', 'libs/image_processing:kaiko-image-processing-reqs#numpy'].

My project structure is as follows:

├── BUILD
├── libs
│   ├── geometry
│   │   ├── BUILD
│   │   ├── pyproject.toml
│   │   └── requirements.txt
│   └── image_processing
│       ├── BUILD
│       ├── pyproject.toml
│       └── requirements.txt
└── pants.toml

And my

root

pattern is:

[source] # <https://www.pantsbuild.org/docs/source-roots>
marker_filenames = ["pyproject.toml"]

My

libs/image_processing/BUILD

file is as follows:

python_requirements(name="kaiko-image-processing-reqs")

python_sources(
    name="kaiko-image-processing",
    sources=["kaiko/image_processing/**/*.py"],
    dependencies=[
        ":kaiko-image-processing-reqs#opencv-python",
        "libs/geometry:kaiko-geometry",
    ],
)

I could list in

dependencies

all external dependencies, like I did for

opencv-python

but my developers push back quite hard on boilerplate, and this will make my life to integrate

pants

harder. Is there a workaround so that

pants

doesn't go looking for dependencies higher up than the enclosing root? I would like it picks up

'libs/image_processing:kaiko-image-processing-reqs#numpy'

without further ado here. (edited)

Hi all, I am currently evaluating pants for our fledgling MonoRepo and liking it so far 🙂 The main challenge I am facing is that we use flutter(frontend) and Pulumi(IaC), which are not (as far as I know) currently supported by pants. What is the most straightforward way to include them in the build process? For Pulumi, I would like it to detect

Pulumi.yaml

files and dependencies between different stacks (happy to describe this in the

BUILD

files) and perhaps do a

preview

on build and an

up

on publish. For flutter (I don't know enough about it) but would like to be able to integrate the build commands (

flutter pub get

,

flutter test

etc., in the relevant goals!) Any help with this would be much appreciated :)

Trying to begin the process of upgrading our Python version and I’ve added

interpreter_constraints=["CPython~=3.7.4", "CPython~=3.10.8"]

to my test utils (and a parametrise for those two versions to the tests:

interpreter_constraints=parametrize(py3_7=["CPython~=3.7.4"], py3_10=["CPython~=3.10.8"])

) and I get this error

pants.engine.process.ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython~=3.10.8,~=3.7.4' failed with exit code 102.
stdout:

stderr:
Could not find a compatible interpreter.

Examined the following interpreters:
1.)   /Users/user/.pyenv/versions/3.7.4/bin/python3.7 CPython==3.7.4
2.) /Users/user/.pyenv/versions/3.10.0/bin/python3.10 CPython==3.10.0
3.) /Users/user/.pyenv/versions/3.10.3/bin/python3.10 CPython==3.10.3
4.) /Users/user/.pyenv/versions/3.10.6/bin/python3.10 CPython==3.10.6
5.) /Users/user/.pyenv/versions/3.10.8/bin/python3.10 CPython==3.10.8
6.)  /Users/user/.pyenv/versions/3.8.13/bin/python3.8 CPython==3.8.13
7.)  /Users/user/.pyenv/versions/3.9.13/bin/python3.9 CPython==3.9.13
8.)   /Users/user/.pyenv/versions/3.9.5/bin/python3.9 CPython==3.9.5

No interpreter compatible with the requested constraints was found:

  Version matches CPython~=3.10.8,~=3.7.4

Is this not how I’m supposed to do it, or is there something else wrong here?

Hi all. I don’t know, I find the Pants documentation about where to put things, and what the syntax looks like, very confusing. I’m trying to add

extra_requirements

so I can get pytest to use the requests module with something like this

extra-requirements=["requests==2.28.1"]

in my

pants.toml

file. But that doesn’t address the problem that the request module isn’t found when I write the test. :( What is the syntax and where does it go?

I’ve got a

[source]

section like this in my

pants.toml

file:

[source]
root_patterns = [
    "/project/app",
    "tests"
]

But my unit test code can’t find my project modules to import, things like this in the test code:

from . import app

How do I configure pants so test code can import project modules?

Hello all, I am new to pants build and I wanted to setup flask project with pants build

I usually create all my project just like that and manage it using docker compose

having each app run in a container

but these days I wanted a monorepo system where I can structure my code in a better way

Is there a pants community forum? I’ve seen the blog, but I’m wondering if there’s a forum where I could also post questions and information to the community.

Hello! I’m using VSCode and I am using the pants flake8 extra_requirements field to bring in flake8 plugins and I would like my vscode to use the same flake8 and plugins when doing IDE level linting. The problem I’m running into is that when I run

./pants export

it is creating a bunch of virtual environments, and I can only point my IDE at one of them at a time. How does one solve this problem and have all my tools and configs synced between vscode and pants?

I've read the pants CI guide and understand how to only run the tests on transitive set of tests that differ from trunk but I'm a little unsure what to do after landing in trunk. Package everything? Package only what has changed since since some elaborate variant of HEAD^1? Package everything and if you are using a cache that actually only packages things that have changed? (Same question for "deploy", but I know that's alpha.)

Hey Pants Team, has anyone ran into the issue of Pants not collecting dependees of removed modules? I noticed this when upgrading from 2.7 -> 2.8 that Pants does not report dependees of removed modules. Where in 2.7 Pants did to report dependees for this.

is it allowed to dynamically infer dependencies for a target which lies outside of the project root where pants is run?

Hello. I’m having difficulties working with the Python protobuf integration, and I’m wondering if anyone can suggest any debugging strategies. If I run the

export-codegen

goal, I see the generated Python files where I expect them under

dist/codegen

. But when `run`ing a script that depends on these generated files, I get an import error. I’ve included

__init__.py

files in every directory. Is there a way to manually inspect the directories that Pants links at runtime, especially the directory where it stores generated code?

most pants commands failing for me with “ModuleNotFoundError: No module named ‘pex.bin’” and

ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython<4,>=3.7' failed with exit code 1.

e.g.

./pants fmt pushdown_service/::

why would this happen? sorry, I can’t share the repo as it’s not open source