Has the idea come up yet to support a trace intrinsic? It would be useful to be able to add something like this to a rule to debug:

trace = await Get(Trace, Here())
print(f"At {here}")

And have a rendering of the stack of rules that are suspended waiting on the current (

Here()

) rule.

heads up! the default https://github.com/pantsbuild/pants branch has been renamed to `main`: all PRs have already been updated, but if you have a source checkout of Pants, you’ll need to rename the branch manually.

FYI just landed Tokio v1.x upgrade, please let me know if you see any issues!

OK, I think I've finally gotten stuck on how to do something in the new engine. I'm running

coursier fetch

, which returns me a report on which jars were fetched and the absolute paths where they were fetched. They're fetched into a subdir of

global_options.options.named_caches_dir

. But now I want to gather up only the jars that were relevant to this resolve into a

Digest

where the jars have been flattened out (or at least have the named cache prefix trimmed off). AFAICT the only way to do this would be with

PathGlobs

, but it only works on paths relative to the build root. Is there some other way to capture arbitrary files from known locations on disk into a

Digest

?

I'm also trying to understand better what a

Workspace

is. Is this like a hermetic temp directory made for each rule execution, or process invocation? Based on poking around at code, I'm having trouble deciding whether it's just for interacting with the equivalent of

dist/

, (i.e. "repo global"), or if it's a sandbox for hermetizing task activity

4210c6c9dece3ff6d0c00b318b880bd6050c3e96

resulted in

./pants lint

hanging with Black, Flake8, and Docformatter - it doesn't finish execution. Could someone please try running

./pants lint src/python/pants/util/strutil.py --no-process-execution-use-local-cache --no-pantsd

?

Any opposition to renaming

--no-process-execution-cleanup-local-dirs

to be

--no-process-execution-cleanup

? If we ever wanted something like that for RE, I imagine it would be

--no-remote-execution-cleanup

Motivation is to make it less painful to type, given that this is so useful to debug. In fact, we might want to suggest both

--print-stacktrace

and

--no-process-execution-cleanup

upon Pants errors?

Semi-related, rename

--no-process-execution-use-local-cache

to

--no-local-cache

, or

--no-process-execution-local-cache

if that's too little context

hey folks: if you have a box with a spinning disk and are able to build Pants on it, would you mind trying out: https://github.com/pantsbuild/pants/pull/11782#issuecomment-805065370

Can someone please run

rm -rf .pids && ./pants

on

main

and see how long it takes for the scheduler to initialize? It's taking 8 seconds for me, which seems like a perf regression. But 2.4.0.dev1 is still taking me 5 seconds 👀

This is likely a pex question, but I’ll post it here anyways, feel free to do with it what you like. Not sure if this is a bug or feature, but I noticed that I could not run a pex file from another directory, only from current dir.

~/src/.../
$ /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY/pylint_runner.pex --version
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY/pylint_runner.pex/__main__.py", line 74, in <module>
  File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY/pylint_runner.pex/__main__.py", line 31, in __maybe_run_venv__
  File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY/pylint_runner.pex/.bootstrap/pex/variables.py", line 689, in venv_dir
  File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY/pylint_runner.pex/.bootstrap/pex/variables.py", line 687, in add_pex_path_items
  File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY/pylint_runner.pex/.bootstrap/pex/pex_info.py", line 99, in from_pex
FileNotFoundError: [Errno 2] No such file or directory: 'pylint.pex/PEX-INFO'
~/src/.../
$ cd /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY
/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY
$ ls
__plugins                            pylint.pex                           pylint_runner.pex_bin_python_shim.sh requirements.pex
__run.sh                             pylint_runner.pex                    pylint_runner.pex_pex_shim.sh        src
/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionQXCoGY
$ ./pylint_runner.pex --version
pylint 2.7.2
...

Seeing this weirdness:

statler:[~/src/pants][fix_urls]$ RUST_BACKTRACE=1 ./pants
10:56:30.54 [INFO] initializing scheduler...
10:56:30.98 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: PyErr { ptype: <class 'AttributeError'>, pvalue: Some("module 'pants.util.docutil' has no attribute 'docs_url'"), ptraceback: None }', src/externs/mod.rs:265
10:56:30.98 [ERROR] Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.
10:56:30.98 [INFO] Filesystem changed during run: retrying `ResolvedPluginDistributions` in 500ms...
10:56:31.49 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: PyErr { ptype: <class 'AttributeError'>, pvalue: Some("module 'pants.util.docutil' has no attribute 'docs_url'"), ptraceback: None }', src/externs/mod.rs:265
10:56:31.49 [ERROR] Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.
10:56:31.49 [INFO] Filesystem changed during run: retrying `ResolvedPluginDistributions` in 500ms...
10:56:31.99 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: PyErr { ptype: <class 'AttributeError'>, pvalue: Some("module 'pants.util.docutil' has no attribute 'docs_url'"), ptraceback: None }', src/externs/mod.rs:265
10:56:31.99 [ERROR] Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.
10:56:32.00 [INFO] Filesystem changed during run: retrying `ResolvedPluginDistributions` in 500ms...
10:56:32.50 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: PyErr { ptype: <class 'AttributeError'>, pvalue: Some("module 'pants.util.docutil' has no attribute 'docs_url'"), ptraceback: None }', src/externs/mod.rs:265
...

Goes on forever

rust v1.51.0 is out, although clippy still fails with the same ICE as in v1.50.0 (and which is fixed on nightly):

Is it possible to emit an output from a rule but effectively force the input key for that output to be something other than the rule's inputs? Concrete scenario: I perform an external resolve, which involves a lot of dependency and conflict resolution and whatnot. But I also learn some more narrowly "keyed' facts: e.g. this exact Maven coordinate corresponds to this exact

Digest

of a downloaded jar. It would be useful if I could inform the engine of that relationship so that later resolves, with different inputs and therefore different keys, could still take advantage of that

coor -> Digest

relationship and potentially avoid refetching

I'm kind of imagining a rule

fetch_single_coordinate(coord: MavenCoord) -> Digest

, except I just tell the engine to act as though that rule has executed and give it both the input and the output

I could also see why maybe this is fundamentally impossible in the engine for good philosophical reasons, since a bug in such a rule could lead to issues with fingerprinting

Does the engine provide a mechanism to get just the

FileDigest

from a file within a

Digest

or

Snapshot

? I see I can get

FileContent

, but I don't actually need the bytes of the file, just the fingerprint

@busy-vase-39202 has left the channel

So I'm hacking on a rule, and I'm now stuck at a point where I see this output ad infinitum:

20:03:30.47 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: "Field `content` was not convertible to type alloc::vec::Vec<u8>: PyErr { ptype: <class \'TypeError\'>, pvalue: Some(\'an integer is required (got type str)\'), ptraceback: None }"', src/intrinsics.rs:354
20:03:30.47 [ERROR] Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.
20:03:30.47 [INFO] Filesystem changed during run: retrying `@rule(pants.backend.experimental.jvm.goals.coursier_resolve.coursier_generate_lockfile)` in 500ms...

has renamed the channel from "engine" to "development"

set the channel description: Discussion around development of Pants itself.

set the channel description: Discussion around development of Pants itself, including issues with the project's CI infrastructure.

Shellcheck's config file support isn't great. It will look for a global ~/.shellcheckrc, along with a repo specific one. Further, there is no way to specify a path to an arbitrary config, so it would have to be at the repo root to work properly with Pants What do folks thing of disabling config files for Shellcheck by always setting its

--norc

option? I'm very concerned about the risk of ~/.shellcheckrc breaking reproducibility, which is a core objective of Pants to provide

Is there no way to turn off just the global one but allow the repo-specific one?

Would source roots make sense for a Shell implementation? If you have an

.sh

file at

src/sh/foo.sh

and it sources

src/sh/dep.sh

, would you expect to do

source dep.sh

or

source src/sh/dep.sh

? I don't think there would be an equivalent to PYTHONPATH, beyond maybe cd-ing into the source root?

Anyone waiting on anything for the dev release?

It looks like the toolchain plugin is using the deprecated

PantsEnvironment

- I'll bump the deprecation for now, but looks like someone should push a new plugin 🙂

The opt-in anonymous telemetry is quite slow. This may have already been known, but I stumbled upon it working on other perf concerns this am. In the Pants repo itself it adds ~550ms latency on my machine / network. Filed https://github.com/pantsbuild/pants/issues/11833

Hm, I've been working on adding Shunit2 to the new Bash plugin for tests, but the below is giving me pause and making me wonder if we should add shunit2 or not..