I didn't expect that to work, and it's only gotten me useful stack traces sometimes

screwing around with things right now and the new ExternalTool and Target APIs are VERY hype

and

ExecuteProcessRequest

=>

Process

was also the right change to make

i am hacking away on a cargo ruleset because a facebook eng working on libra messaged me earlier today and asked me if i'd heard of their library for understanding and manipulating cargo projects: https://github.com/facebookincubator/cargo-guppy and i think this is exactly what we would need to be able to meaningfully manipulate cargo projects without having to literally run cargo in a chroot (which is what i did before, intentional tech debt)

i won't go too far into the weeds without trying to checkpoint things into a PR

i'm actually very excited that this could make the cargo ruleset extremely small

a repl for executing

await Get[X](Y(...))

statements would make me drool right now but dropping in a

pdb.set_trace()

is enough for right now

When running master with

./pants

, I’m getting this change automatically:

diff --git a/src/rust/engine/Cargo.lock b/src/rust/engine/Cargo.lock
index 310ba1f86..cb05e4b4e 100644
--- a/src/rust/engine/Cargo.lock
+++ b/src/rust/engine/Cargo.lock
@@ -3126,6 +3126,7 @@ source = "registry+<https://github.com/rust-lang/crates.io-index>"
 dependencies = [
  "bytes 0.5.4 (registry+<https://github.com/rust-lang/crates.io-index>)",
  "fnv 1.0.6 (registry+<https://github.com/rust-lang/crates.io-index>)",
+ "futures-core 0.3.4 (registry+<https://github.com/rust-lang/crates.io-index>)",
  "iovec 0.1.4 (registry+<https://github.com/rust-lang/crates.io-index>)",
  "lazy_static 1.4.0 (registry+<https://github.com/rust-lang/crates.io-index>)",
  "libc 0.2.69 (registry+<https://github.com/rust-lang/crates.io-index>)",

@hundreds-father-404: that would be colliding rust changes (one PR does X, another does Y)… should commit it

Cool. Included in https://github.com/pantsbuild/pants/pull/9718

I am seeing this on latest master... I looked the the code... it feels weird... what am I missing ? https://github.com/pantsbuild/pants/blame/be07ae749f785bd07bf389fe6d9aa847926d8d90/src/python/pants/engine/collection.py#L33-L38

https://github.com/bheisler/criterion.rs i would love love love a performance regression testing infrastructure for pants

the haskell library it’s inspired by lists as a feature:

Measurements are cross-validated to ensure that sources of significant noise (usually other activity on the system) can be identified.

which sounds like something i’ve literally never seen on any benchmark before

Did the rust / cargo version get updated in the last few months? I cherry picked a pretty recent commit and I’m getting this error:

error[E0658]: `&std::sync::Arc<InvalidationWatcher>` cannot be used as the type of `self` without the `arbitrary_self_types` feature
   --> watch/src/lib.rs:256:28
    |
256 |   pub async fn watch(self: &Arc<Self>, path: PathBuf) -> Result<(), notify::Error> {
    |                            ^^^^^^^^^^
    |
    = note: for more information, see <https://github.com/rust-lang/rust/issues/44874>
    = help: consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)

Its some kind of new experimental feature of rust, but I dont see a macro to enable it anywhere on master.

In the process of cherry-picking a bunch of stuff onto the 1.25.x-twtr branch I ran into an issue that I can repro on my local master. Steps to repro are 1.

git checkout master

2.

./v2 fmt ::

3.

Ctrl-c

while that is running. For me the pants process is unkillable with Ctrl-c. I bisected a version of this issue on my local 1.25.x-twtr branch and it seems to be introduced in https://github.com/pantsbuild/pants/pull/9674 If someone could give this repro a shot on theit machine, I will make an issue for it once its confirmed not just my machine.

@hundreds-father-404 would we be interested in moving the

.expect_single()

method into

Collection

instead of into

Targets

?

this is really cool!!

has anyone seen this error when using

Box::pin(async move { ... }).compat()

? It's not clear to me why I'm being told this future is not `Send`:

error[E0277]: `dyn futures_io::if_std::AsyncRead + std::marker::Unpin` cannot be sent between threads safely
   --> src/intrinsics.rs:350:4
    |
350 |   .to_boxed()
    |    ^^^^^^^^ `dyn futures_io::if_std::AsyncRead + std::marker::Unpin` cannot be sent between threads safely
    |
    = help: the trait `std::marker::Send` is not implemented for `dyn futures_io::if_std::AsyncRead + std::marker::Unpin`

for the code:

fn fetch_cargo_packages(context: Context, args: Vec<Value>) -> NodeFuture<Value> {
  let lockfile_contents: String = externs::project_str(&args[0], "contents");
  let cargo_home_dir: String = context
    .core
    .cargo_fetcher
    .download_dir
    .to_str()
    .unwrap()
    .to_string();

  let core = context.core.clone();
  Box::pin(async move {
    let _krates: Vec<_> = core
      .cargo_fetcher
      .fetch_packages(&lockfile_contents)
      .await?;
    Ok(externs::unsafe_call(
      &core.types.construct_fetched_cargo_packages,
      &[externs::store_utf8(&cargo_home_dir)],
    ))
  })
  .compat()
  .to_boxed()
  .map_err(|e: cargo_utils::CargoMetadataError| throw(&format!("{:?}", e)))
  .to_boxed()
}

it also provides this error, on the exact same line:

error[E0277]: `dyn futures_io::if_std::AsyncWrite + std::marker::Unpin` cannot be sent between threads safely
   --> src/intrinsics.rs:350:4
    |
350 |   .to_boxed()
    |    ^^^^^^^^ `dyn futures_io::if_std::AsyncWrite + std::marker::Unpin` cannot be sent between threads safely
    |
    = help: the trait `std::marker::Send` is not implemented for `dyn futures_io::if_std::AsyncWrite + std::marker::Unpin`
    = note: required because of the requirements on the impl of `std::marker::Send` for `&mut dyn futures_io::if_std::AsyncWrite + std::marker::Unpin`

note the

AsyncRead

and the

AsyncWrite

are both causinig an error

more:

= note: required because it appears within the type `std::future::GenFuture<[static generator@src/intrinsics.rs:339:23: 348:4 core:std::sync::Arc<context::Core>, lockfile_contents:std::string::String, cargo_home_dir:std::string::String for<'r, 's, 't0, 't1, 't2, 't3, 't4> {context::Core, std::sync::Arc<context::Core>, &'r cargo_utils::CargoPackageFetcher, cargo_utils::CargoPackageFetcher, std::string::String, &'s str, &'t0 std::string::String, impl core::future::future::Future, impl core::future::future::Future, (), ()}]>`
    = note: required because it appears within the type `impl core::future::future::Future`
    = note: required because of the requirements on the impl of `std::marker::Send` for `std::ptr::Unique<impl core::future::future::Future>`
    = note: required because it appears within the type `std::boxed::Box<impl core::future::future::Future>`
    = note: required because it appears within the type `std::pin::Pin<std::boxed::Box<impl core::future::future::Future>>`
    = note: required because it appears within the type `futures_util::compat::compat03as01::Compat<std::pin::Pin<std::boxed::Box<impl core::future::future::Future>>>`
    = note: required because of the requirements on the impl of `boxfuture::Boxable<core::Value, cargo_utils::CargoMetadataError>` for `futures_util::compat::compat03as01::Compat<std::pin::Pin<std::boxed::Box<impl core::future::future::Future>>>`

i've tried moving this part into a separate function

but it seems that calling this method with

.await

is causing the problem

i don't know what

AsyncRead

is or why it's not

Send

, or how to make it

Send

You could typecast to Send. I’ve had to do that in some non-Pants rust code

so

as Box<T + Send + whatever else>

it was actually with AsyncRead

so cast the Box with

as Box<dyn AsyncRead + Send>