The

secrets

on

docker_image()

is not on 2.9.0.dev4?

Hi, sorry to pile more on the docker train, but noticing a few issues there myself. 0) minor bug:

pants tailor

will create both

python_sources()

and

docker_image()

thus creating a name conflict if it sees a Dockerfile and python code in the same location. 1) Have

[python] interpreter_constraints = [">=3.9"]

, but somehow the pex_binary target, when placed inside the docker image, calls

python3.8

(and fails, because I'm using a 3.9 image). 2) In my situation the entrypoint is defined through a third-party helm chart which needs to know about the target python package to import. Hence I only need a python env with my code in the image. I found using

PEX_TOOLS=1 ./archive.pex venv ...

works perfectly with the corresponding

include_tools

option in

pex_binary

. Except that https://pex.readthedocs.io/en/v2.1.57/recipes.html#pex-app-in-a-container refers to env vars like

PEX_PYTHON=3.9

, which appear to not do anything (like patch the issue (1))

Is there a way to find out whether or not the transitive dependencies for a single target have changed compared to

origin/master

?

I want to have certain jobs/steps in my CI (github actions) run only if dependencies for a specific file have changed

Hi folks! If you want to show some quick love to the Pants project, there are a couple of easy ways to do so: 1) Star the repo (and encourage your Pants-using teammates to do the same) 2) Add your company to the list of Pants-using orgs (after getting any internal permission you need) This helps grow the project, which in turn benefits us all! So thanks in advance!

A question about dependencies. I have a

3rdparty/requirements.txt

file with the

python_requirements()

in the BUILD sister file. so, for example:

❯ ./pants dependencies 3rdparty:boto3
3rdparty:requirements.txt

Now suppose I change the version of boto3 in

requirements.txt

.

./pants list --changed-since=HEAD --changed-dependees=transitive

will return, well, basically everything. One some level this makes sense (everything has a 3rd party requirement), but how can I filter on just those files that are affected by the change to that single requirement?

OK, excited about the docker functionality in 2.8 and got it to do... most of my containers. But I'm having a little pickle with a few which rely on private repos. I was able to get docker-buildkit functionality working with

pants.toml

and

[docker]
env_vars=["DOCKER_BUILDKIT=1", "SSH_AUTH_SOCK"]

...so that I could get

mount=type=ssh

working for ssh authentication, and that's where we go a bit wrong. Running under pants, dockerfiles with

RUN --mount=type=ssh

don't seem to work, even though in the generated

__run.sh

includes the required

#!/bin/bash
# This command line should execute the same process as pants did internally.
export DOCKER_BUILDKIT=1 SSH_AUTH_SOCK=/tmp/path/to/ssh/agent/sock
cd /tmp/process-directory-tmp
/usr/bin/docker build -t $'my-tag' -f my-dockerfile .

if I

cd

to that directory and run

__run.sh

, everything builds. If I

pants package my-docker-target

, which is what generated that file, it doesn't build and gives an ssh key error when downloading custom requirements from a private github repo. Anything we can do about that? Am I missing a flag or is this a problem with the docker backend?

I am noticing that in CI caching

~/.cache/pants/setup

doesn't cache any external pants plugins (global.plugins) and those get downloaded on every CI run. any idea which directory should be cached in order to avoid the re-download in every CI run. I found no reference to that in https://www.pantsbuild.org/docs/using-pants-in-ci (note that we use remote caching, so we nuke lmdb_store and named_caches before uploading the data to the CI cache)

Is there a way to lock down (using a lock file with hashes) the python packages that get installed via global.plugins ?

I have looked at the https://www.pantsbuild.org/v2.9/docs/rules-api-tips#debugging-visualize-the-rule-graph which helps in visualizing the graph (which is helpful when developing a plugin). However, I can’t seem to find a way to produce the dependency graph of a target, in a similar fashion that Bazel can? See https://docs.bazel.build/versions/4.2.2/tutorial/cpp.html#review-the-dependency-graph.

Howdy. I'm trying to run pants 2.8.0 with

interpreter_constraints = ["CPython==3.10.*"]

. I'm on ubuntu 18.04 (bionic) and my python3.10 binary is from the deadsnakes ppa. This is what I get:

ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython==3.10.*' failed with exit code 102.

stdout:

stderr:

Could not find a compatible interpreter.

Examined the following interpreters:

1.) /usr/bin/python2.7 CPython==2.7.17

2.) /usr/bin/python3.6 CPython==3.6.9

3.) /usr/bin/python3.8 CPython==3.8.0

4.) /usr/bin/python3.9 CPython==3.9.9

(That 3.9.9 version is from deadsnakes as well. If I specify CPython==3.9.*, it's picked up and everything's fine). So, is my python3.10 installation broken somehow, or not supported?

Should I prefer granular per-dir build files, or one top-level one with big fat globbing. 🧵 1. First option is what

tailor

would have me use. One

BUILD

file in each directory that matters. ◦ Pros: Granular. Pants internals kind of assume this pattern (so target names are sane, etc...). ◦ Cons: Still boilerplate-y (MUCH less so than say, Bazel. But still have N files which just repeat

python_sources()

,

python_test_utils

,

python_tests

over and over). 2. Second is one

BUILD

file in the "root" which has nested globs for each (E.g.

sources=["***/**.py", "!**/conftest.py", ...]

) and then I can override when necessary at the directory level (assuming that works 🤔 ) ◦ Pros: Minimal boilerplate ◦ Cons: Less granular. Pants internals don't assume this (the target name of the generator is funny looking when printed) Q: Is pants smart enough for option 2 to not have double ownership? Can it be taught to be? Q: What considerations should be made about cache invalidation? If I edit the top-level file in #2, do I bust any caching? Q: For option 2, can I improve the names that get spit out? Does the "generators-are-not-targets" proposal help this case?

Hi. To learn more about Pants, I was considering trying to implement a plugin that adds importlinter(https://import-linter.readthedocs.io/en/stable/index.html) as a new linter. But I’ve realized that importlinter is different from existing pants linters, in that it expects to be invoked in the root of the project with all code visible, so that it can detect whether any import contracts have been violated. Other Pants linters can be freely invoked on only a subset of files in a project. Does this mean that it maps poorly onto the Pants concept of a linter? Would it be better represented as its own goal?

Is it possible/makes sense to use the

--cache-from

on the

docker_image

?

Hello! I want to download and save nltk stopwords but I'm not sure how to achieve this using pants. For more information, I have this BUILD file and I would like to make sure stopwords are also available for this particular project. I would like to do something like this or similar.

# src/python/projectA/BUILD

python_sources(
    dependencies=[
        "//:nltk",
    ]
)

python_distribution(
    name="wheel",
    dependencies=[":projectA"],
    provides=setup_py(
        name="projectA",
    ),
    wheel=True,
)

Can someone help point me to the right direction? Thanks so much.

Hello - I am interested in dynamically creating docker tags. I can generate them using git commands .. something like

"$(git log -1 --date=unix --format=%cd)-$(git rev-parse --short HEAD)"

. I see there is a note about this requiring custom plugin implementation (https://www.pantsbuild.org/docs/tagging-docker-images#tagging-images). Is there any existing examples or other places I should look before attempting to implement something? Thanks!

We have a pre-built

.whl

file which we have added to our repo because it is particularly complex to build from source. This works fine if I use an absolute path as the requirement but can not find a way to make it work with relative paths.

python_requirement(
    name="assimulo",
    requirements=[
        "assimulo@ file:////path/to/3rdparty/python/wheel/Assimulo-trunk-cp38-cp38-linux_x86_64.whl"
    ],
)

python_requirement(
    name="assimulo",
    requirements=[
        "assimulo@ wheel/Assimulo-trunk-cp38-cp38-linux_x86_64.whl"
    ],
)

Is there a known path forward to use a relative path to a .whl as a requirement?

hello, i’m looking into integrating cython with pants. i’ve seen a couple previous questions in this channel about the topic and wonder if there’s any updates with that. the project currently uses

pyximport

to build the simple library rather than using setuptools - is it possible to configure pants to support this workflow without a custom plugin?

I am failing to produce a PEX using a wheel that contains some extended metadata in

Requires-Dist

section that may confuse

pip

; thread has details.

A poll: which do people prefer for the default "resolve" and lockfile for your Python code? A "resolve" is a logical name for a lockfile, and it's what will allow multiple lockfiles in your project. If you want some code to use a different lockfile than the default, you'll set

resolve="data_science"

for example. A. Abbreviate for less verbosity:

{"py-default": "3rdparty/py/default_lock.txt"}

B. Use full name for clarity:

{"python-default": "3rdparty/python/default_lock.txt"}

Is there a way to use pants to list all the tests (not test targets)?

Perhaps it would be useful to have a “parallellism group” option for tests, where a set of tests can be grouped, and an option telling how many of those that may run in parallell, as I think that running too many in parallell seems to saturate available resources.

Hello everyone! Quick question: I have poetry dependencies in a pyproject.toml (at root level), my target depends on

python-magic

and the module it exposes is just

magic

. I checked the Module Mapping docs, however I don't understand how to use it. I expect to declare in the root BUILD file that such mapping exists

poetry_requirements()

python_requirement(
    module_mapping={"python-magic": ["magic"]}
)

But I get a MappingError

Targets in root-level BUILD files must be named explicitly.

Hi Team, Trying to use vcrpy with pytest and pants to have HTTP interactions recorded and replayed in my tests. I can see the cassette output in the pants test directory - but I need it to be output in the current dir to be commited to repo + I need the cassette when running the tests to prevent network hit when the tests are played on the CI. I have a feeling I need to write a plugin for that

is that right?

Hi everyone, I’m trying to run this command

./pants test ::

but I got the following error :

18:27:01.07 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython>=3.8' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/pex.py", line 476, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/pex.py", line 532, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/pex.py", line 668, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/pex.py", line 699, in execute_pkg_resources
    runner = entry.resolve()
  File "/Users/arth/.cache/pants/named_caches/pex_root/unzipped_pexes/01a8d22400b26b1d132a9849fb6e28962f1ecf9e/.bootstrap/pex/vendor/_vendored/setuptools/pkg_resources/__init__.py", line 2481, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/arth/.cache/pants/named_caches/pex_root/installed_wheels/7fea57708be4117c2d962e01abd1585a8846dc06/pex-2.1.54-py2.py3-none-any.whl/pex/bin/pex.py", line 30, in <module>
    from pex.pex import PEX
  File "/Users/arth/.cache/pants/named_caches/pex_root/installed_wheels/7fea57708be4117c2d962e01abd1585a8846dc06/pex-2.1.54-py2.py3-none-any.whl/pex/pex.py", line 12, in <module>
    from distutils import sysconfig
  File "/Users/arth/.pyenv/versions/3.8.10/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 92, in create_module
    return importlib.import_module('setuptools._distutils')
  File "/Users/arth/.pyenv/versions/3.8.10/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'



Use --no-process-execution-local-cleanup to preserve process chroots for inspection.

I ran on Mac M1, and it worked before 😕 But I don’t know why now it can’t find setuptools. does this ring a bell to someone ?

Hey y'all, we upgraded from

2.8.1rc1

to

2.9.0.rc0

yesterday. Noticed today that the pex files that end up getting deployed do not have executable permissions. The permissions are: •

2.8.1rc1

->

-rwxr-xr-x

•

2.9.0.rc0

->

-rwxr--r--

I've tried packaging locally and we end up with the correct permissions, seems the pex files created by our CI (Github, ubuntu-latest) are missing those two executable permissions. Any idea on what might cause this or how to remedy this?

(Wait nevermind. I'm going to try and install-from-git-VCS first)