And relatedly, after trying out Sam Gross'

nogil

fork of CPython 3.9.10, and then building the engine with his

nogil

fork of PyO3 I see... A lot of seg faults and other errors. So, we'll have to hold our breath for that one. Presumably, eventually, we'd get a

nogil

interpreter we can leverage.

Do we think the adhoc_tool improvements could change the equation on whether Pants could become purely Rust because fewer people need to write plugins? I know this is a wildcard idea

FYI PR to add an auto-greeter to newcomers to the repo: https://github.com/pantsbuild/pants/pull/18977

Do we have interest in trying out the "stale issue" GHA? https://github.com/actions/stale By default (but configurable): • Add a label "Stale" on issues and pull requests after 60 days of inactivity and comment on them • Close the stale issues and pull requests after 7 days of inactivity • If an update/comment occur on stale issues or pull requests, the stale label will be removed and the timer will restart

👋 what're the long-term goals behind the macro system? specifically, are they intended to eventually be shareable with other Pants users (like Bazel macros)?

I'm upgrading to Pants 2.16 and seeing something I wasn't expecting. Surely I'm missing something:

$ tree . -L 1
.
├── __filter.py
├── __nbconvert.py
├── notebook.pex
├── notebook.pex_bin_python_shim.sh
├── notebook.pex_pex_shim.sh
├── notebooks
├── __pycache__
├── __run.sh
└── __template.py.j2

3 directories, 6 files

In

./__run.sh

export PEX_MODULE=__nbconvert
cd /tmp/pants-sandbox-EuAiqL
./notebook.pex_pex_shim.sh nbconvert ...

Running it I get

$ ./__run.sh 
Traceback (most recent call last):
  File "/tmp/pants-sandbox-EuAiqL/.cache/pex_root/venvs/8af8dff5caba8ff4e3555dafecebf663b2d3b550/3ae15bb9525f8e6ffaefc76519921e604f1e7079/pex", line 274, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/home/josh/.cache/pants/named_caches/pyenv/44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a/versions/3.8.16/lib/python3.8/runpy.py", line 203, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/home/josh/.cache/pants/named_caches/pyenv/44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a/versions/3.8.16/lib/python3.8/runpy.py", line 138, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named __nbconvert

So the only change I would expect to make a difference is PEX version?

Perhaps, but from the info you provided this should not work. Did you leave out a PEX_EXTRA_SYS_PATH or similar from the __run.sh contents?

@bitter-ability-32190: it looks like the

pyo3

upgrade caused wheel building shards to start failing on `main`: e.g. https://github.com/pantsbuild/pants/actions/runs/4943093059/jobs/8837254210

So GitHub has an option we can enable which gives people a one-button "Merge latest

main

into this branch". Since our classifier script requires your branch is within some static number from

main

, I suggest we enable this option to make the bring-to-the-present operation simple

PR for 2.16.0rc2 prep: https://github.com/pantsbuild/pants/pull/18986

can someone re-run this job https://github.com/pantsbuild/pants/actions/runs/4958844263/jobs/8872222241?pr=18978 ? (I don't have permissions)

hey folks: i’m on release duty today. i’ll execute any `rc`s / finals i can from

2.14.x

and

2.15.x

once the ARM shard is healthy again. but i’m going to hold off on

2.16.x

and

2.17.x

releases, since Benjy did/is-doing them in the last few days. i’ll aim to do a

2.16.0

final early next week, and cut the

2.17.x

branch at the same time

For transparency, here's the plan to "migrate" the django dep inference code: • Instead of plugging into the new union (which I hope to remove), plugin into the normal dep inference union • To make it performant, only infer deps from files that "look" like migrations (under a "migrations" folder with a filename like ...) • Then do rudimentary AST traversal/scraping That will allow us to remove the existing dep inference union/code for Python wholesale in 2.18

there is a

pantsd

crash on the Linux-ARM64 shards on

main

, but i am completely out of steam for investigating it today. maybe Monday.

OK folks, restart your AARCH64 shards and we'll see

This test flakes often on my opened PR:s src/python/pants/pantsd/service/store_gc_service_test.py:tests

Anyone have opinions on per-target kob on ignore/warn/error of unmatched source(s) glob?

hmmm... something missing ? (everything works... this 404 doesn't seem to break stuff)

it looks like something broke hitting the “native binaries” cache/artifacts on

main

, so test shards currently rebuild the rust code from scratch.

OK, one thing I've noticed about

CODEOWNERS

is that the codeowner is still there even if reviewers are added. IMO I should've known that in hindsight. So, I'm starting to be -1 on the change as that wasn't the intent. Instead I might suggest we: • Migrate this to a GitHub Action -> "if there are no reviewers on a PR, add X, Y, or Z." Could still make it based on path. • Add a PR template which in a comment suggests to the PR author to leave it blank if they are unsure and someone will be added automatically Thoughts?

I have some compiled code in repo and I want to use vanilla Pants as well as dep inference (why I have this in a plugin today). 1. I can't find the docs or the code to imply that this is already supported by Pants, is that right? 2. Is this something we could do? Have like

modules

on

python_distribution

and add a

FirstPartyPythonMappingImplMarker

union impl?

There has been a noticeable uptick in folks pinging about Windows support and offering to help the Pex effort here. I've let folks know the deal there, basically I'm a bottleneck getting the Pex Windows branch green since I have a hard line on green CI or bust, and I've pointed out folks interested in helping can help eliminate other interfering work - namely Pex fixes / features. Well, I thought about this for 5 minutes and quickly realized there is a very obvious way to help somehow folks have missed - get CI setup for Windows for Pants for the engine. IIUC the only change there would be a skip for the native client IT tests that actually run Pants. Those aside, getting the full rust codebase clippy + test pass in Windows CI has just been hanging out there for anyone to do.

How can I test a rule_helper? RuleRunner doesn't appear to have a way to invoke one by name

Hey folks, could I please get a review on https://github.com/pantsbuild/pants/pull/18974 ?

Why does the

AdhocProcessRequest

construction specify the

shell_command.address.spec

after the command? https://github.com/pantsbuild/pants/blob/d12d7e2026225159219b21aba159bd0b6c3f8d39/src/python/pants/backend/shell/util_rules/shell_command.py#L147 So it looks like:

run_shell_command(
    name = "foo",
    command=f"echo hi",
    workdir="/",
)

turns into:

/usr/bin/bash -c $'echo hi' $'//:foo'

ooh, this is quite cool: https://pyo3.rs/main/contributing.html#documenting-changes (tl;dr: require a user-readable snippet per PR, and use something like

towncrier

to concatenate them)

Are there any backends that have to deal with file mtime and/or mode? It looks like the

FileContent

/

Digest

is similar to

git

in that it only preserves the executable bit - the mtime and other bits of file mode are not accounted for. For some package types (I'm working on rpm/deb/etc), mode and mtime are significant as they are baked into the package. So, I'm trying to think of a sane way to define defaults for mode / mtime, and I'm hoping that some other backend (or pex?) has already had to deal with this and what solution was used.

Which UX do you prefer for a target with a

file_mode

field (for file permissions/mode)? • int:

file_mode=0o644

• str:

file_mode="644"

• object:

file_mode=perm(u=6, g=4, o=4)

• Something else? Ultimately this will end up represented as an octal in yaml, when passing it to the underlying app.

For the GIL bottleneck thing, what stops us from running multiple interpreters via the engine and collecting their results in the graph? I'm assuming someone has thought of this and there are good reasons we don't or can't do it, but I'm curious what they are

we’ve previously discussed moving from an “every SHA is consumable with PANTS_SHA” to “only commits which have been tagged or workflowed would be consumable with PANTS_SHA”. i think that doing that would align well with some automation that i’m looking at in the release process. how would we feel about that? it would mean that a maintainer would need to run a workflow to build wheels for a SHA.