#17483 Twine env vars not exposed properly New issue created by kaos Describe the bug Using the

TWINE_

env vars without a repository suffix does not work without explicitly adding them to the subprocess env using a configuration like:

[subprocess-environment]
env_vars.add = ["TWINE_PASSWORD", "TWINE_USERNAME"]

Pants version

main

OS Any. Additional info As reported by @Kludex on Slack: https://pantsbuild.slack.com/archives/C0D7TNJHL/p1667741059650449 pantsbuild/pants

#17486 Coarsened target calculation for `pylint` pulls in more files than it should on v2.15.0a0 Issue created by danxmoran Describe the bug While testing

v2.15.0a0

, I saw that

./pants lint

would consistently freeze / be OOM killed. Through

py-spy

and logging I found that: • Much time was being spent merging together source digests when setting up

pylint

runs • The number of source digests in each of ^^^ setups was much larger than expected (tens of thousands of files for an input batch size of a few hundred) •

pylint

batches of different sizes always ended up with the same number of source digests While looking through the

pylint

changes in v2.15, I found that coarsened target calculation is currently running in the "partitioner" rule (see here). This will result in too many targets being associated with each

pylint

batch, because the partitions returned by that rule are re-batched into smaller chunks according to

[lint].batch_size

, and there's no support in the re-batching logic for subsetting the partition metadata. We should push the calculation of coarsened targets into the "runner" rule for

pylint

, so we only compute & hydrate the transitive dependencies that are relevant for the specific inputs used in each batch. Pants version v2.15.0a0 OS Both pantsbuild/pants

#16354 Allow customization of registry when running a docker image Issue created by thejcannon Is your feature request related to a problem? Please describe. When I

./pants run myservice:img

it uses the registry name as part of the docker tag, which is annoyingly long 🙈 E.g.

<http://123456789987654321.dkr.ecr.us-west-1.amazonaws.com/myservice:latest|123456789987654321.dkr.ecr.us-west-1.amazonaws.com/myservice:latest>

is now on my list of

docker images

. Describe the solution you'd like Allow me to customize the registry when just `run`ning my image. E.g.

docker_image(
    registries=[...],  # for package and publish
    run_goal_registry="companyname",
)

which then

./pants run myservice:img

would make

companyname/myservice:latest

Describe alternatives you've considered I have this in a plugin, but I don't like turning off the standard plugins. #15359 could help the pain though... Additional context N/A pantsbuild/pants

#16827 link `.syso` files into package archive Issue created by tdyas Go allows packages to provide precompiled object files with a

.syso

extension to be linked directly into the package archive as-is. The current Pants support does not do this but should in order to properly support building Go packages. There are also some complications for this support when a package supports cgo (which will need some investigation into source code). See https://github.com/golang/go/blob/54182ff54a687272dd7632c3a963e036ce03cb7c/src/cmd/go/internal/work/exec.go#L3027-L3042 for details in

go

toolchain. pantsbuild/pants

#17497 Please add Ponder to the public list of Pants Users Issue created by csnatarajan Company name Ponder Company website https://ponder.io/ Company logo No response pantsbuild/pants

#10271 remoting: support symlinks attribute in directories Issue created by tdyas #10155 refreshed the REv2 protos. The

DirectoryNode

proto now has a `symlinks` field that models symbolic links in a directory as a list of `SymlinkNode` protos. Pants should implement support for that in order to better conform to the REv2 API. pantsbuild/pants

#17502 Rust backend New discussion created by rbuckland Preamble This is a discussion thread to gather the requirements, ideas and wisdom for creating a development backend workflow for Rust. This is intended to be a bit of a brain dump and request for ideas/opinions/thoughts on creating a compilation and packaging workflow for the C and C++ backend(s). Currently only "firing" shell command is supported, no rust. • ...src/python/pants/backend This discussion thread is to flesh out the themes we would need, such that we can incrementally build out a Rust backend. Base principle is to no reinvent any wheel, but let a rust project, live within in a wider polyglot monorepo (thing a Rust lib that generates Python, C and npm (webpack) libs) Required Features • Resource/Dependency Generation (bin or lib is output) • Compilers/Toolchains • Package Management • Linting and Formatting • Testing • Compilation (inc multi-arch support) • Caching • Build Generators Resource/Dependency Generation • w.r.t. Dependency Inference: • Is reading the

cargo.toml

enough to determine what will be "spat out" - need to know the architectures being generated of course. Compiler and Toolchain The assumption would be that we can hook straight into cargo execution Do we run rustc ? Package Management cargo.lock file exists, - need to look how it is handled With Python/pip,poetry and Java/maven today. This should be similar same, which has the best example ? Linting and Formatting run cargo fmt :-) ? Testing run cargo test :-) ? Compilation • same , run cargo. • What about debug output formats etc • multi architecture generation Caching need to know how the caching architecture works and how we seed it from a rust point of view. Build Generators • w.r.t. Dependency Inference: • In theory this may just be, have extension points to that a call to "shell" can declare what will be output. An example (personally used) is auto generation of json-schema files, from the rust struct external data types. Then used in APIs. Actions • collate the above list into a clear set of requirements • call out any dependencies or "solved by one action" between them • identify any known gaps in Pants today that blocks us for beginning this work pantsbuild/pants

#16833 cgo: add `cgo_enabled` field to `go_binary` to control cgo support Issue created by tdyas Currently Cgo support is controlled by the

--golang-cgo-enabled

flag. Allow cgo support to instead be controlled on a package-by-packge basis via a

cgo

boolean field on

go_package

with additional fields to add C flags etc., and a

pure

boolean field on

go_binary

which disables cgo support entirely for that binary. This is similar to what Bazel offers with rules_go. pantsbuild/pants

#17503 Thank you. Issue created by kaos

Thank you.

This problem seems loosely related to the idea of "hygienic macros": i.e., that a macro should be able to define anonymous variables which can't collide with or pollute the original namespace. But I don't know how to do that and still have consumers later depend-on/find them... maybe by having the request for the syntethic target use something other than its name/address? In any case: shipping to unblock the fix. Originally posted by @witty-crayon-22786 in #17365 (review) pantsbuild/pants

#17504 Hygienic synthetic targets New discussion created by kaos Stu:

This problem (from #17365) seems loosely related to the idea of "hygienic macros": i.e., that a macro should be able to define anonymous variables which can't collide with or pollute the original namespace. But I don't know how to do that and still have consumers later depend-on/find them... maybe by having the request for the synthetic target use something other than its name/address?

Originally posted by @witty-crayon-22786 in #17365 (review) My idea being we could open up a discussion about synthetic target hygiene. I know @Eric-Arellano wanted the synthetic targets to live in their own namespace, or something like that. While the synthetic targets needs to exist also in the regular target namespace (to be able to being target for dependencies and extension by BUILD file targets), perhaps it would be useful to be able to reach also shadowed synthetic targets in some way? pantsbuild/pants

#1977 Is it possible to find the correct --platform argument on a computer without pex? Issue created by michaelochs Hey. I am trying to create a pex bundle to then put this on a computer that does not have pex nor pip installed, basically all it has is a basic python interpreter. I keep running into issues where I get error messages along the line of

The wheel tags for PyNaCl 1.5.0 are XXX which do not match the supported tags of XXX

It does not list which tag it requires. Is there any way to either get these platform identifiers out of the python runtime or have a very basic pex file that I can copy over and execute to find which string I should pass to

--platform

to get the pex file to work there? Thanks for your help. pantsbuild/pex

#16854 experimental_shell_command outputs do not preserve symlinks Issue created by huonw Describe the bug Any symlinks in the

outputs

list of

experimental_shell_command

are resolved to normal files, duplicating the contents of the thing they link to. It'd be nifty to have the option to preserve them. At the very least, it'd be good to document this behaviour. There's some cases where being a symlink (rather than a normal file) is be semantically relevant. For instance, with node, it seems the files within

node_modules/.bin

are symlinks into the rest of

node_modules

, and turning a link into a normal file causes those executable scripts to stop working. Consider the following `BUILD`:

experimental_shell_command(
    name="make-link",
    command="""
    echo normal > normal.txt
    ln -s normal.txt link.txt
    # double check
    test -L link.txt
    """,
    tools=["ln"],
    outputs=["*.txt"]
)

experimental_run_shell_command(
    name="check-link",
    command="""
    cd {chroot}
    ls -l
    test -L link.txt || echo "not a link!!"
    """,
    dependencies=[":make-link"]
)

The key is

make-link

, while

check-link

is just a way to be able to introspect what

make-link

splats out when used as a dependency. The output of

./pants run //:check-link

is:

total 16
-rw-r--r--  1 huon  staff  7 14 Sep 12:02 link.txt
-rw-r--r--  1 huon  staff  7 14 Sep 12:02 normal.txt
not a link!!

But I'd expect it to not have the

not a link!!

printed, and the

ls

line for

link.txt

should be

lrwxr-xr-x ... link.txt -> normal.txt

. Full reproducer https://gist.github.com/huonw/4ebc739a7a04cdd9bcf137bdcbc2018f:

git clone <mailto:git@gist.github.com|git@gist.github.com>:4ebc739a7a04cdd9bcf137bdcbc2018f.git
cd 4ebc739a7a04cdd9bcf137bdcbc2018f
./pants run //:check-link

Pants version 2.13.0 2.14.0rc0 OS macOS Additional info Thanks for pants! 👖 pantsbuild/pants

#17509 `CoarsenedTargets.closure()` returns duplicate targets Issue created by danxmoran Describe the bug While investigating

pylint

perf regressions in 2.15.x, I found that this call to `closure` is yielding duplicate elements (sometimes 10s of thousands of duplicates in our very tangled repo). @witty-crayon-22786 confirmed it should not be doing that. Pants version 2.15.x In 2.14.x the

closure

call for

pylint

wrapped the result in a

FrozenOrderedSet

, so the bug could be present there too OS Both pantsbuild/pants

#17511 Dependees does not return files dependent on removed modules Issue created by njgrisafi Describe the bug When removing a whole Python module Pants`dependees` does not return dependent files. This seems to be a behavior change between version 2.7 -> 2.8. I also tested on the latest version 2.14 and it seems to be a warning now:

10:28:00.15 [WARN] Pants cannot infer owners for the following imports in the target app/module_1/example.py:

  * app.module_2.example.test (line: 1)

For example,, given this module:

app/
  module_1/
      __init__.py
      example.py (imports module_2.example)
      BUILD
  module_2/
      __init__.py
      example.py
      BUILD

When removing

app/module_2/

, add changes in git and then running

./pants dependees --changed-since=HEAD

does not return

app/module_1/example.py

in dependee results. Pants version 2.7 (last known working version) 2.8 (first known version with issue) 2.10 (current version) OS Both MacOS and Linux Additional info Please see example repo with predefined version branches: • https://github.com/njgrisafi/pants-example/tree/27 (2.7) • https://github.com/njgrisafi/pants-example/tree/28 (2.8) Checkout one of the follow version branches above and perform the following test: 1.

rm -rf app/module_2/

2.

git add --all

3.

./pants dependees --changed-since=HEAD

Confirm the dependees output between versions. 2.7 should return

module_1

and 2.8 will not. pantsbuild/pants

#17512 Dependees (dependents) should return dependent files for removed modules New issue created by njgrisafi Is your feature request related to a problem? Please describe. Related to this bug report I opened but changed it into a feature request. When removing a whole Python module

dependees

(now

dependents

) does not return dependent files. On the latest version 2.14 and it seems to be a warning:

10:28:00.15 [WARN] Pants cannot infer owners for the following imports in the target app/module_1/example.py:

  * app.module_2.example.test (line: 1)

On earlier versions it returns no output. Describe the solution you'd like When removing modules and then running

./pants dependees --changed-since

(or

dependents

), dependent targets of the removed modules will be returned in the output. For example, given this module:

app/
  __init__.py
  module_1/
      __init__.py
      example.py (imports app.module_2.example)
      BUILD
  module_2/
      __init__.py
      example.py
      BUILD

When removing

app/module_2/

, add changes in git and then running

./pants dependees --changed-since=HEAD

it should return

app/module_1/example.py

in the results. Describe alternatives you've considered An additional layer of tooling over pants to detect these issues. Custom logic to account for this. Open to more ideas. Additional info Please see example repo with predefined version branches: • https://github.com/njgrisafi/pants-example Checkout one of the follow version branches above and perform the following test: 1.

rm -rf app/module_2/

2.

git add --all

3.

./pants dependees --changed-since=HEAD

pantsbuild/pants

#17517 naming an target's field for Go race detector New discussion created by tdyas I would like feedback on what to name a new field on various Go target types to enable the Go data race detector. Relevant PR is #17510. Bazel rules_go just uses

race

for the name of their equivalent concept. Some proposals: •

race

•

with_race

•

with_race_detector

And test-only variants: •

test_race

•

test_with_race

•

test_with_race_detector

pantsbuild/pants

#17519 remoting: support `{chroot}` in remote execution context New issue created by tdyas The Pants local and Docker command runners support substituting the string

{chroot}

in an argument or environment variable with the absolute path of the execution sandbox. This is done via the `apply_chroot` function. Pants should support the same substitution in remote execution as well. This will require use of a wrapper script since the remote execution environment's paths will not be known until the build action is actually invoked. A wrapper script for other purposes (append-only caches) was added by #17290 and thus it should be relatively straightforward to do this substitution. pantsbuild/pants

#1885 venv_dir calculation doesn't correctly handle PEX_PYTHON_PATH with symlinks. New issue created by hrfuller I noticed a bug in the variables.venv_dir hash computation. If PEX_PYTHON_PATH has the path of a symlinked interpreter that interpreter value isn't added to the hash of the venv_dir. This can cause collisions in which interpreter is used to resolve dependencies via resolver.resolve. This happens because the canonical path to the interpreter is computed but the PEX_PYTHON_PATH is assumed to have canonical paths. line in question I think it would be reasonable to map the PEX_PYTHON_PATH entries to their realpath's before comparing with the requested venv interpreter. pantsbuild/pex

#17521 When getting started with the JVM, it's not obvious how/when to create your intial lockfile New issue created by stuhood To create a lockfile for the JVM, you need to run

./pants generate-lockfiles

. But if you haven't done that, and have redefined the lockfile location, you will get a unmatched globs warning instead:

10:14:46.00 [ERROR] 1 Exception encountered:

  Exception: Unmatched glob from The resolve `jvm-default` from `[jvm].resolves`: "lib/3rdparty/jvm/default.lock"

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.14/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.

Rather than an unmatched glob, we should instead detect the missing file at a higher level, and explain how to create the lockfile. It looks like we already attempt to do that (in some cases): pants/src/python/pants/jvm/resolve/coursier_fetch.py Lines 761 to 767 in </pantsbuild/pants/commit/c1514b73a6248a366228549c95bc1cc00fc2d93c|c1514b7> ... but apparently not in others. pantsbuild/pants

#17526 Mypy cache is shared across all repos using pants New issue created by huonw Describe the bug Per https://pantsbuild.slack.com/archives/C046T6T9U/p1668121011324399, the new append-only mypy cache has a single instance per python version, and thus potentially incorporates information across repositories that makes the cache inefficient. For instance, on my machine:

tree ~/.cache/pants/named_caches/mypy_cache/               
# /Users/huon/.cache/pants/named_caches/mypy_cache/
# ├── 3.7
# │   └── cache.db
# ├── 3.8
# │   └── cache.db
# └── 3.9
#     └── cache.db
# 
# 3 directories, 3 files

# a sample of 30 random paths:
sqlite3 ~/.cache/pants/named_caches/mypy_cache/3.9/cache.db 'SELECT path FROM files ORDER BY random() LIMIT 30'

The sample of the paths includes things like

pants/backend/codegen/export_codegen_goal.data.json

(from the pants repo),

tabulate/__init__.meta.json

(from a dependency), and paths from our internal repo source roots. Pants version 2.14 OS macOS Additional info N/A pantsbuild/pants

#17531 Issue resolving generated Python module with source at root New issue created by dlsmith Describe the bug Python failed to resolve a generated module (in this case, from protobuf) when the project was structured with the Python source at the root of the package. My hypothesis is that this is due to the way Python resolves modules: it looks first in the execution directory, then to

PYTHONPATH

. Pants links the generated code from the sandbox via the

PYTHONPATH

, but if it replicates/preserves the semantics of Python, the interpreter may first look in a file tree with the parent module structure but without the generated files, leading to an import error. I've included a minimal example below that demonstrates the issue and works around it by nesting the source root under a

src/

prefix. Pants version 2.14.0 OS MacOS Additional info pants-codegen-repro.tar.gz pantsbuild/pants

#17533 ./pants fails to bootstrap if PY is not set New issue created by tdyas When bootstrapping a new clone of the Pants repository at latest ,

./pants

failed with the following error message because

PY

was not set:

paradox:~/Projects/Pants/pants% ./pants
/REDACTED/pants/build-support/pants_venv: line 20: -c: command not found
Bootstrapping pants_deps with requirements:
  /REDACTED/pants/3rdparty/python/requirements.txt
/REDACTED/pants/build-support/pants_venv: line 20: -c: command not found
/REDACTED/pants/build-support/pants_venv: line 20: -c: command not found
/REDACTED/pants/build-support/pants_venv: line 33: : command not found

Failed to create venv.

Setting

PY=python3

allows bootstrap to work. (

pyenv

was already configured with CPython 3.10 as

python3

). pantsbuild/pants

#17536 Tests using tensorlfow fail on M1 Mac (Fatal Python error: Illegal instruction) New issue created by kamioka-masaya Describe the bug Tests using tensorlfow fail on M1 Mac.

BUILD

python_requirement(
    name="tensorflow",
    requirements=[
        # Linux or intel Mac
        "tensorflow == 2.10.0 ; sys_platform != 'darwin' or platform_machine == 'x86_64'",
        # Mac M1
        "tensorflow-macos == 2.10.0 ; sys_platform == 'darwin' and platform_machine == 'arm64'",
    ],
)

Pants version 2.14.0 OS MacOS Additional info ``` ============================= test session starts ============================== platform darwin -- Python 3.8.12, pytest-7.0.1, pluggy-1.0.0 -- XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/bin/python3.8 cachedir: .pytest_cache rootdir: /private/var/folders/ql/XXX/T/pants-sandbox-HM09tq plugins: xdist-2.5.0, forked-1.4.0, mock-3.10.0, cov-3.0.0 collecting ... Fatal Python error: Illegal instruction Current thread 0x0000000204ec7600 (most recent call first): File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 1166 in create_module File "<frozen importlib._bootstrap>", line 556 in module_from_spec File "<frozen importlib._bootstrap>", line 657 in _load_unlocked File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/tensorflow/python/platform/self_check.py", line 63 in preload_check File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 26 in <module> File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 843 in exec_module File "<frozen importlib._bootstrap>", line 671 in _load_unlocked File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 36 in <module> File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 843 in exec_module File "<frozen importlib._bootstrap>", line 671 in _load_unlocked File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 961 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 37 in <module> File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 843 in exec_module File "<frozen importlib._bootstrap>", line 671 in _load_unlocked File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 961 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "/private/var/folders/ql/XXX/T/pants-sandbox-HM09tq/src/python/lib/media_analyzer/ml/model/tensorflow/builder_test.py", line 2 in <module> File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 171 in exec_module File "<frozen importlib._bootstrap>", line 671 in _load_unlocked File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 991 in _find_and_load File "<frozen importlib._bootstrap>", line 1014 in _gcd_import File "XXX/.pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127 in import_module File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/pathlib.py", line 533 in import_path File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/python.py", line 599 in _importtestmodule File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/python.py", line 510 in _getobj File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/python.py", line 296 in obj File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/python.py", line 527 in _inject_setup_module_fixture File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/python.py", line 513 in collect File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/runner.py", line 371 in <lambda> File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/runner.py", line 340 in from_call File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/runner.py", line 371 in pytest_make_collect_report File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39 in _multicall File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80 in _hookexec File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265 in call File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/runner.py", line 544 in collect_one_node File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/main.py", line 825 in genitems File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/main.py", line 657 in perform_collect File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/main.py", line 332 in pytest_collection File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39 in _multicall File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80 in _hookexec File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265 in call File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/main.py", line 321 in _main File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/main.py", line 268 in wrap_session File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/_pytest/main.py", line 315 in pytest_cmdline_main File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39 in _multicall File "XXX/.cache/pants/named_caches/pex_root/venvs/s/XXX/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80 in _hookexec File "XXX/.cache/pants/named_caches/pex_root/venvs/s/… pantsbuild/pants

#17539 Build docker image for specific platform New issue created by huonw Is your feature request related to a problem? Please describe. A docker image can be targeted to a specific platform, e.g.

linux/amd64

will run on x86-64 Linux systems, while

linux/arm64

will run on AArch64 ones. This can be configured via the

--platform

argument to

docker build

, and defaults to the current platform: https://docs.docker.com/engine/reference/commandline/build/#options . This means that an uncustomised

docker build

on ARM (e.g. macOS with Apple Silicon) will build for

linux/arm64

. It'd be nice to be able to control the platform for a

docker_image

, so that, for example, docker images that are deployed on

amd64

can be built for that platform, independent of the host. Currently, we're having to have two complete platforms JSON specs (one for amd64 and one for arm64), and build our dependency PEX for both (following https://blog.pantsbuild.org/optimizing-python-docker-deploys-using-pants/ and/or https://pex.readthedocs.io/en/v2.1.113/recipes.html#pex-app-in-a-container), which ~doubles the size of the resulting PEX, takes longer to download from a cache, and takes longer to manipulate. Describe the solution you'd like A

platform

(or similar) argument to

docker_image

that is passed through as

docker build --platform ...

. Describe alternatives you've considered I don't know of one. Additional context N/A pantsbuild/pants

#17540 Sandbox must be disabled for VS Code remote debugging to work Issue created by rhinck Describe the bug Filed by request from @thejcannon When running

./pants run --debug-apapter {path_to_target}

and using Visual Studio Code's

Python: Remote Attach

feature, VS Code is able to attach to the remote debug server but breakpoints are never hit when Pants sandboxes are used. Breakpoints only work when the Pants sandbox is explicitly disabled. Pants version 2.14.0 OS macOS (haven't tested on Linux) Additional info • Slack Thread • Example project pantsbuild/pants

#17541 Post-landing CI goals New discussion created by cburroughs I've read the pants CI guide https://www.pantsbuild.org/docs/using-pants-in-ci and understand how to only run the tests on transitive set of tests that differ from trunk. That's great! But I'm a unsure what the typical thing to do is once the code has landed in trunk: Package everything? Package only what has "changed" with some elaborate variant of HEAD^1? Package everything and if you are using a fine grained cache that actually only packages things that have changed? (Same question for "deploy", but I know that's alpha.) pantsbuild/pants

#17542 `experimental_shell_command` can trigger consistent cache misses Issue created by chrisjrn While working on #16825, we found a case where an

experimental_shell_command

target is consistently not cached. To reproduce: 1. Clone https://github.com/danxmoran/pants-yarn-install-repro 2. Run

./pants_from_sources -ldebug run :run-yarn-install

3. Look for:

Error loading process execution result from local cache: Computed digest for Snapshot

It appears as though the bug was introduced between be00ed5 and 952235a. pantsbuild/pants

#17543 Deprecation warning about InteractiveProcessRequest while using publish for Docker images Issue created by Gaasmann Describe the bug When using the publish target for pushing Docker images, I see the following warning in the logs:

13:08:28.42 [WARN] DEPRECATED: InteractiveProcessRequest is scheduled to be removed in version 2.15.0.dev1.

 

Instead, use `InteractiveProcess.from_process`.

Pants version Which version of Pants are you using? 2.14.0+gitfae1f69f (the pex file on the GH release page) OS Are you encountering the bug on MacOS, Linux, or both? Linux, I can't test other OSs. pantsbuild/pants

Comment on #17541 Post-landing CI goals Discussion answered by benjyw Great question. It really depends on your org's practices of course, but if you have a lot of services, and don't want to redeploy them constantly, then you can use something like, e.g.,

./pants list --filter-target-type=pex_binary --changed-since=release_branch --changed-dependents=transitive

to find all the pex_binary targets affected by changes since the release_branch. This assumes your deploys are tagged in git, which is very good practice. pantsbuild/pants

#16999 Write metadata about packaged docker image to dist/ Issue created by huonw Is your feature request related to a problem? Please describe. Currently there doesn't seem to be an easy way to determine info about a docker image that was built via

./pants package some/path:some_docker_image

(and/or published via

./pants publish ...

). This is unfortunate when the image repositories and/or tags are dynamic, or if one wants to use the image ID. In particular, machine use of this info (e.g. using it in terraform/cloudformation/... templates) seems to require parsing the human-focused output of the pants commands. There's currently no record of a docker image being packaged in

dist/

, since the image itself is managed/stored by docker itself. This was discussed in slack at https://pantsbuild.slack.com/archives/C046T6T9U/p1663916571660779. Describe the solution you'd like There was a suggestion of writing out a JSON file like

dist/some.path/some_docker_image.docker_info.json

that contains metadata about the image, effectively acting as a "link" to the compiled artefact. For example:

{
    "repositories": ["example.repo"]
    "image_tags": ["pants-hash-123456789", "latest"]
    "image_id": "1234567890"
}

Questions: • should this be versioned somehow? • anything else to include? • is there a different format that may work better? (e.g. I imagine it may be common to want to use a shell script to interpret this output) Describe alternatives you've considered None, yet. Additional context #14657 may be tangentially related, since I could imagine it may result in layers being written out to

dist/

(maybe?). pantsbuild/pants