Let's not abuse the stable release system for stuff like this

Hello! I’m trying to build Pants on an M1, and I’ve got the latest master, so even running under rosetta, it’s using the correct venv. Currently it looks like Rust is consistently building under the wrong architecture:

chrisjrn@chrisjrns-MacBook-Pro pants % lipo -info /Users/chrisjrn/src/pants/src/python/pants/engine/internals/native_engine_pyo3.so
Non-fat file: /Users/chrisjrn/src/pants/src/python/pants/engine/internals/native_engine_pyo3.so is architecture: arm64

Hey, for our FFI port, I want to implement the trait FromPyObject on the PathGlobs struct defined in Rust. It would allow our FFI functions to request that type in function signature and do the right thing, rather than having to request a &PyAny But the struct is defined in a different crate, so rust doesn't allow adding a trait Instead, the two options seem to be: 1) create a light weight wrapper struct. Implement Deref on it so it's accessed the same, right? 2) use a free function rather than the trait. This is what we currently do. Thoughts on which is preferable?

I'm unaware of a real problem being reported in the wild, but it struck me that our means of doing resolves - Python and JVM - is broken when using constraints ranges due to Process use and the resulting caching. Once resolved there is no reasonable way to ask for a re-resulve to get newer versions except by mutating requirements strings: https://github.com/pantsbuild/pants/issues/12199 Lockfile can solve this, since re-generating a lockfile will bump to newer versions, but a lockfile is a required thing.

@hundreds-father-404: should i take a swing at https://github.com/pantsbuild/pants/issues/12019, or would you like to look at it?

Hm I wish that we did our metric differently for remote cache writes. Rather than

remote_cache_write_{started,finished,errors}

, I wish we did

{start,success,error}

. That is,

finished

is currently a superset of

errors

, and it would be more useful for analysis if it was disjoint How would we want to approach API stability for metrics? Note that it's a good time to make this change now because they weren't working how we wanted anyways https://github.com/pantsbuild/pants/issues/12019

at some point while i was out on leave (or perhaps when i updated xcode yesterday), Pants’ own integration tests started picking up a broken (?) Python 3.8 install

but also wondering whether other folks have strategies for passing through their settings for configuring python interpreters. i have:

$ cat ~/.pants.rc
[python-setup]
# Avoid system python.
interpreter_search_paths = ["<PYENV>"]

…which works for the outer run, but which is currently (intentionally) disabled in inner/test runs

In v1,

interpreter_constraints

on a

pex_binary

target would override the transitive closure. In v2, we "fixed" it to instead act like all other interpreter constraints, to be mixed in. The practical impact is described in https://www.pantsbuild.org/docs/python-interpreter-compatibility#using-multiple-python-versions-in-the-same-project

This means that every dependency of a target must also be compatible with its interpreter constraints.

Thoughts if that was a mistake or not?

Hi! I’m working on https://github.com/pantsbuild/pants/issues/12059 and I want a quick sanity check — would an option to watch the filesystem belong in the local store options, or is it more of an independent thing?

Hey can someone please pull

main

and run

./cargo fmt

? Do you get changes?

I'm trying to understand a Pants target/rule API behavior here; I'm not sure if this is deliberate and I'm misunderstanding the intent, or if it's not how Pants is supposed to behave. So I have a very basic

Target

subclass, JavaLibrary. When I use the

java_library

alias in a BUILD file with two Java sources present (using the default pattern to capture all non-test

.java

sources), I expect a single

JavaLibrary

target to be produced in the build graph. But instead, there are 3 targets:

//:lib

, which depends on

//ExampleLib.java:lib

and

//OtherLib.java:lib

(corresponding to the two source files). I can sort of see based on my past experience with Pants why we might be generating "synthetic" targets in this situation to more cleanly encapsulate source files, but here's the catch: the two "synthetic" targets depend on each other (circularly). This doesn't appear to directly upset the build graph, but it does cause my rule to break since it introduces a cycle in the rule graph. Am I using this API wrong, or is this just something that hasn't been an issue for Pants yet because the Python rules don't have to distinguish between direct and transitive dependencies much?

hah, just found a use for a

future::select_all

that preserves the order of the remaining tasks, and was reminded that i pointed out that it shouldn’t in the first place =S

How do I increment a counter from code that does not have a context available? (Such code would have to call `workunit_store::get_workunit_store_handle`but given

increment_counter

was moved to the workunit, unclear how to model this situation. Would the code just call

in_workunit!

and increment the counter in a trivial child workunit?

Yep. Or adjust to pass it in.

@witty-crayon-22786 re auto-generating lockfiles. Let's say you change

[black].version

then run `./pants fmt`: how important do you think it is that

black_lock.txt

gets regenerated, vs. something like warning or erroring that you should run

./pants lock

? I'm wondering if the latter is adequate so that we can punt on secondary effects in downstream rules: https://github.com/pantsbuild/pants/issues/12014. Trying to keep the scope of lockfile project tight

looks like i’m on release duty for Friday: it sounded like we were ready for a

2.6.0rc0

?

@bored-art-40741: so, the recently removed nailgun code was https://github.com/pantsbuild/pants/commit/e19336db0d3df6361e08c53b921da7c07a880045

Does the pants repo have an opinion or precedent for adding

README.md

files within nested subfolders where such documentation is warranted? I'm to the point on some projects where I need to write some prose and long examples as long-term documentation, and it likely doesn't belong in code comments/docstrings. On the other hand, I'm not sure that the user facing docs website is the best place either, since it's still an experimental backend and the documentation is very much into the weeds about the backend itself. Any thoughts?

Any reason not to promote 2.5.1rc3 to 2.5.1 today?

https://developers.facebook.com/blog/post/2021/07/01/future-of-buck/ More build systems taking on rust as a core language :) curious about what their extensibility offering will look like!

@hundreds-father-404: one more thought re: lockfile names: if we really need to try and support the exact same lockfile format/filename someone could probably symlink from whatever name we choose to the location that poetry is expecting

I've been wondering about the following: Today if

python_library

X depends on a

python_distribution

Y then X effectively depends on all of the libraries Y depends on, transitively, as sources, and nothing special happens with Y itself. You could replace it with its direct deps with no change in behavior. But what the author of that dep probably intended is for X to depend on the wheel built from Y. An example of where this is useful is when you have a

python_distribution

that builds native code via a custom

setup.py

(this is now possible! See https://github.com/pantsbuild/pants/pull/12250). AFAICT today we handle this as a special case, only for running tests, via

runtime_package_dependencies

. But it seems like we should do this generically?

were people familiar with mozilla’s sccache (https://github.com/mozilla/sccache) for distributed compilation? i just learned about it a few minutes ago and it touches a lot of use cases (configuring build servers to connect to, doing hermetic compiles and providing a toolchain, supporting rust) that are relevant to our interests. in terms of things we can use it for directly (https://github.com/mozilla/sccache/blob/master/docs/DistributedQuickstart.md), i think the most immediate would be introducing a rust toolchain to hermetic process executions in a structured way (i know rust language support is not an immediate priority, but this is essentially the first blocker to starting that i think). i think it’s likely we could steal a few ideas from the project regarding configuration, process scheduling, as well as object storage, although i’m not sure yet what method it uses for that last one.

I'm about at the end of my ability to debug this CI issue, so hoping someone here recognizes the issue or has suggestions on how to debug further: https://github.com/pantsbuild/pants/pull/12283/checks?check_run_id=3002558607

did everyone else know about PEP 517 already? https://www.python.org/dev/peps/pep-0517/

I would like to extend the

pex_binary

target docstring to comment on packaging multiplatform PEX files. https://github.com/pantsbuild/pants/blob/f9b5815352cb571e98f3214a23b83c86f96ee2a7/src/python/pants/backend/python/target_types.py#L242 So the idea is that it is possible to create multi-platform PEX files so that the very same

.pex

file can be distributed and run both on Linux and MacOS device. This is achieved by adding multiple platform tags for the

pex_binary

target declaration. This will make sure that the

.pex

file contains wheels for both platforms (e.g., if

numpy

is a requirement, then there will be one

.whl

file for

numpy

compiled for MacOS and one

.whl

file for

numpy

compiled for Linux). If there are many requirements with compiled non-Python code, a multi-platform PEX file can be significantly larger than a platform-specific one. It wasn’t clear to me that multiple wheel files will be included (one for each target platform) which became obvious once I have unzipped the

.pex

file and inspected the

.deps

directory. Do you think that it’s worth adding this to the docstring? Do you like this wording or it could be expressed in a better way?

--- a/src/python/pants/backend/python/target_types.py
+++ b/src/python/pants/backend/python/target_types.py
@@ -241,7 +241,9 @@ class PexPlatformsField(StringSequenceField):
     help = (
         "The platforms the built PEX should be compatible with.\n\nThis defaults to the current "
         "platform, but can be overridden to different platforms. You can give a list of multiple "
-        "platforms to create a multiplatform PEX.\n\nTo use wheels for specific "
+        "platforms to create a multiplatform PEX. A multiplatform PEX file will contain platform "
+        "specific wheels for each 3rd party requirement that has some non-Python "
+        "code that requires compilation. \n\nTo use wheels for specific "
         "interpreter/platform tags, you can append them to the platform with hyphens like: "
         'PLATFORM-IMPL-PYVER-ABI (e.g. "linux_x86_64-cp-27-cp27mu", '
         '"macosx_10.12_x86_64-cp-36-cp36m"):\n\n  - PLATFORM: the host platform, e.g. '

I’m looking for a way to translate a unparsed target address into a target. Looking at https://github.com/pantsbuild/pants/blob/main/src/python/pants/engine/addresses.py#L41 seems it should be straight forward, but it complains with:

Exception: Could not find a rule to satisfy Get(Targets, UnparsedAddressInputs, UnparsedAddressInputs(values=('//src/python/foo/qux:bin',), relative_to='src/python/foo')).

What could I be missing?

Anyone got anything in flight they're hoping to get into tomorrow's dev release?

tip (hack?) for development in pantsbuild/pants: if you run with

--loop

,

pantsd

will not restart itself, even when you’ve edited the code of pants itself.