pantsbuild #development

Channels

welcome

github-notifications

development

ancient-vegetable-10556

02/14/2023, 6:56 PM

I got annoyed trying to make all of my help strings use

{SomeField.alias}

and discovering that I had circular references, so for your edification, I give you
HelpString
. It looks like an
f
-string, but can be used to forward-refer to classes defined in the same file: https://github.com/pantsbuild/pants/pull/18246

help

Target

and

Field

subclasses now supports 0-argument lambdas, as long as you use the

help_text

helper function for help values! https://github.com/pantsbuild/pants/pull/18248

❤️ 1

curved-television-6568

02/14/2023, 7:52 PM

not sure if this is an issue to report, or in

pants

scie-pants

if so.. 🧵

careful-address-89803

02/15/2023, 6:11 PM

Hey, can a union member itself be a union base? eg

MyLockfileSentinel <: SimplePexLockfileSentinel <: GenerateToolLockfileSentinel

? And would that even work if I had rules that were

(GenerateToolLockfileSentinel) => A

and

(SimplePexLockfileSentinel) => A

sparse-lifeguard-95737

02/15/2023, 10:04 PM

would anyone object to me cherry-picking https://github.com/pantsbuild/pants/pull/18258 back to 2.15.x? IMO it’s very low-risk / a valuable fix, but I know it’s late in the game 😬

shy-architect-61291

02/15/2023, 11:17 PM

Asking to learn -- I started to work on this issue and successfully moved

docker

out to its own crate. However, there are calls in the

<http://docker_tests.rs|docker_tests.rs>

file that reference the

<http://local_tests.rs|local_tests.rs>

file from

process_execution

crate. Despite having scoped the

<http://lib.rs|lib.rs>

file and

Cargo.toml

correctly (as indicated by other dependencies that check out), this will fail

/.cargo test -p docker

with a

Copy code

use process_execution::local_tests::named_caches_and_immutable_inputs
                       ^^^^^^^^^^^ could not find `local_tests` in `process_execution`

Process::new(owned_string_vec(&["/bin/echo", "-n", "foo"]))
         ^^^ function or associated item not found in `Process`

error. For reference, relevant code snippets:

Copy code

// in docker/Cargo.toml
[dependencies]
...
process_execution = { path = "../process_execution" }

// in process_execution/lib.rs
pub mod local;

#[cfg(test)]
pub mod local_tests

// also an import problem:

pub struct Process { /*..*/ };

impl Process {

#[cfg(test)
pub fn new()....
}

Is there something special about test configurations that prevent cross-crate references?

witty-crayon-22786

02/16/2023, 12:00 AM

@happy-kitchen-89482: with about 85% confidence, https://github.com/pantsbuild/pants/issues/18162 does not block

2.15.x

: see https://github.com/pantsbuild/pants/issues/18162#issuecomment-1432238187

careful-address-89803

02/16/2023, 3:06 AM

Hey folks, for exporting python user resolves, we preface the path with the python version. But in the source we have a comment

Copy code

# TODO: It is pretty ad-hoc that we do add the interpreter version for resolves but not for tools.
#  We should pick one and deprecate the other.

Would anything bad happen if we were to not add the full version? I don't think it's possible for a resolve to have 2 interpreters associated with it? removing the version from the path of the exported venv would help solve issues setting up dev environments, like this one, and also what I hit today where my system's version of python got bumped but the old venv stuck around. For backwards compatibility, we could add it as a toggle. or go all the way with the linked issue and provide custom export paths optionally with the version

bitter-ability-32190

02/16/2023, 9:13 PM

Looking at a failure for

tests/python/pants_test/init/test_plugin_resolver.py

due to symlinking big files...

Copy code

# Kill the repo source dir and re-resolve. If the PluginResolver truly detects exact
        # requirements it should skip any resolves and load directly from the still intact
        # cache.
        safe_rmtree(repo_dir)

Can someone help me understand what's going on here? The issue is it the second go-around I guess it's re-resolving plugins (which I suppose is a symptom of the bug) and failing to find

jake

broad-processor-92400

02/17/2023, 1:43 AM

I was playing around with

--loop

for monitoring file changes (very nice when it works), but discovered a bunch of bugs: https://github.com/pantsbuild/pants/issues?q=is%3Aissue+is%3Aopen+loop+infinite I filed them as mostly separate issues, since they felt like they might have different causes/resolutions if one was to play whack-a-mole and fix things individually (since, e.g., test failures under

pants --loop test ...

don't get rerun infinitely, so clearly it's doing something special), but potentially there's a single change that'll make `--loop`'s behaviour much nicer globally?

👍 1

sparse-lifeguard-95737

02/17/2023, 3:06 PM

raising here since I’m not sure who’s seen https://github.com/pantsbuild/pants/issues/17856 we keep bumping into problems with old dependencies (that can’t be updated for Reasons™️). we’ve already isolated those deps in their own resolves, but we don’t have a good way to exclude those resolves at CI time. discussion in the issue suggests there’s some desire for generic filter-by-field functionality, which would solve this issue but seems like it needs a lot of design work. as a short-term, python-specific alternative I had the idea of introducing a new field to the

python

subsystem. something like:

Copy code

python.resolves_to_tags]
scientific = ["intel-only"]
legacy = ["intel-only", "py36-only"]

to have tags auto-apply to all targets in a given resolve. this would solve our problem because once the tags are in place, we can filter on them using existing mechanisms. what do people think? I can try to knock it out if there are no objections 🧵

average-vr-56795

02/17/2023, 9:01 PM

Can I grab a release prep boop? https://github.com/pantsbuild/pants/pull/18297 (I decided not to a0 as 2.15 still seems to be maturing...)

🙏 1

❤️ 1

🚢 1

proud-dentist-22844

02/18/2023, 5:09 PM

@witty-crayon-22786 RFR https://github.com/pantsbuild/pants/pull/18260 I've approved this one, but another pair of eyes would be good before merging.

fast-nail-55400

02/20/2023, 9:15 PM

@enough-analyst-54434: When using

scie-pants

with the

pants

repo itself, what is the correct answer to:

Copy code

The `pants.toml` at /home/tdyas/Projects/Pants/pants/pants.toml has no `pants_version` configured in the `GLOBAL` section.
Would you like set `pants_version` to the latest stable release? (Y/n):

✅ 1

broad-processor-92400

02/20/2023, 10:56 PM

When I request PR reviews, knowing very little about who's got relevant knowledge where, I'm assuming Github's suggested reviewers are an acceptable start?

bitter-ability-32190

02/21/2023, 1:45 AM

Given a PEX, what's the incantation to know what the latest compatible Python version would be? E.g. Given

foo.pex

how could I tease out that

3.8

is the latest compatible

enough-analyst-54434

02/21/2023, 1:50 AM

Just look at PEX-INFO.

unzip -qc foo.pex PEX-INFO | jq .

- it should be pretty obvious from there even if there are no ICs declared.

👀 1

bitter-ability-32190

02/21/2023, 3:54 PM

PexRequest.__init__

is missing a few type hints, which means we have some naughty callers we aren't typechecking.

bitter-ability-32190

02/21/2023, 4:41 PM

Why does the

find_python

rule in

system_binaries

not just use

sys.executable

? For

pants

it'd be the standlone, for

./pants

it'd be whatever the script found.

shy-architect-61291

02/21/2023, 7:54 PM

Hey team, opened this PR to address this Issue today. There's more that can be done, but looking for some feedback re: approach before continuing down the path any further. tl;dr Split the

process_execution

crate up so docker and remote (with remote_cache) are separate. I did not move nailgun since there’s already a nailgun crate in the parent engine directory

❤️ 1

bitter-ability-32190

02/21/2023, 9:51 PM

Yay I have a PoC of Pants providing Python via Pyenv!

Copy code

josh@cephandrius:~/work/pants$ which python3.7
josh@cephandrius:~/work/pants$ pyenv versions
* system (set by /home/josh/.pyenv/version)
  3.8.16
josh@cephandrius:~/work/pants$ ./pants run src/python/pants/runme.py
/home/josh/.cache/pants/named_caches/pex_root/venvs/s/9ccfd9b4/venv/bin/python3.7

Note that Pyenv and building Python was completely sandboxed. (Python is still required on the system for a few non-user-code places, but that can be tackled separetly)

🎉 5

bitter-ability-32190

02/21/2023, 10:04 PM

(totally not related /s) When we make a

VenvPex

it has some guards against the venv being deleted, but if I delete the venv dir I see

path/to/venv_dir/pex: No such file or directory

Working backwards, when we make the PEX with

--venv --seed verbose

I see

pex

in the output JSON having the right path, but the second portion isn't related to any portion in

PEX-INFO

. Where is that second hash coming from?

shy-architect-61291

02/22/2023, 6:34 PM

I’m trying to run (docker, process_execution) tests and all but two panic with the error

“Error making env for store at <dir>: Too many open files”

I see the containers spin up correctly in docker desktop. Following the error shows it originates from the lmdb setup. Is there some config I need to set up prior? (Running MacOS x86_64)

✅ 1

clever-toothbrush-13797

02/23/2023, 1:20 PM

Is there an easy way to make a

pants

release for testing locally, so I might avoid a mistake like https://github.com/pantsbuild/pants/pull/18342 in the future?

proud-dentist-22844

02/23/2023, 4:21 PM

@witty-crayon-22786 RFR https://github.com/pantsbuild/pants/pull/18273

🚢 1

bitter-ability-32190

02/23/2023, 4:26 PM

Copy code

Thread 2 "tokio-runtime-w" received signal SIGSEGV, Segmentation fault.

🫠 From

pants --no-pantsd help

on a branch with only-Python changes

fast-nail-55400

02/23/2023, 9:11 PM

@witty-crayon-22786 / @happy-kitchen-89482: re final release of 2.15.0, does the "What's New" section look good?

ancient-vegetable-10556

02/23/2023, 10:48 PM

I’ve done some very draft quality code to implement runnable dependencies for

shell_command

and

adhoc_tool

, which also allows version pinning for pre-installed binaries that themselves need runnable dependencies (e.g.

yarn

depends on

node

). Would appreciate some feedback on the API approach here: https://github.com/pantsbuild/pants/pull/18347

average-vr-56795

02/24/2023, 12:31 AM

@witty-crayon-22786 I happened to notice https://github.com/pantsbuild/pants/pull/18327 (well, you happened to mention me 😄) - are you aware of docker user namespaces? I've seen this used in CI environments - tl;dr is: In your docker container, you run things as root (user 0), but you use user remapping so that the in-container root user is your out-of-container executing user - it's a really handy way of avoiding permissions errors e.g. requiring sudo to clean-up stuff that happened in the container - as long as everything in the container runs as a predictable user (doesn't have to be root, but often is), you end up with exactly-as-you-want out-of-container permissions

busy-vase-39202

02/24/2023, 7:24 PM

This is cool! Do we have something like this setup? https://twitter.com/megfdev/status/1629131653560913921 I don't recall getting an automated congrats on my first PR, but that would have certainly been a nice moment to give kudos, introduce folks to the page about Contributors and Maintainers role, the Team page, etc.

witty-crayon-22786

02/24/2023, 7:45 PM

is it time!? https://github.com/pantsbuild/pants/pull/18357

😍 4

👖 2