Ergonomic, hermetic, user-friendly software build system for Python, Go, Java, Kotlin, Protobuf, Scala, Shell. Pants lets you fearlessly scale up your codebase!

Pants

I was suggesting Eric to change `support/ivy/ivysettings.xml` to see if anything did not pick up `build-support/ivy/remote.ivysettings.xml`

it looks like a bunch of unit tests are also bootstrapping it, and they just won't be affected by any of the integration test settings anyway. oy

The remote settings should still apply. We use an env var to set `PANTS_CONFIG_FILE=pants.remote.ini` for both shards

the reason we're bootstrapping things is that the `~/.cache/pants` dir is not present for remote execution

presenting it would involve roughly 1) creating a `files` target covering the cache, 2) adding that dependency to all tests that need the cache, 3) redefining the cache dir in tests via an env var

Wouldn’t this kill reproducibility of tests? Any single change to the cache dir would invalidate the test result 

The bin directory is pretty stable. But yeah, that's another reason to opt for explicit

ok.... i think i have a good incremental gameplan for this.
1. add an `extra_env_vars` arg to `python_test` (to mirror jvm) and use it to set the cache dir to a relative path inside the hermetic sandbox
2. add a `files` target shaped like the relevant cache entry
3. add a hacky `curl` in the pants script to populate the input to the `files` target

The only changes to `src` would be part 1, right?

With 3, keep in mind that CI uses pants.pex and not ./pants

the last part is, i think, the only hacky bit. but it could probably be replaced by a "curl a file" target type in the medium term.

yep. the pex-in-the-test-workspace is the inspiration for 3.

When / where would step 3 happen during CI?

right where the conditional pex creation happens

(Flight is about to lift off, talk later about this. Thank you for working on improving this all!)

i'm going to start at the top and work downward.

...slightly insane thought. what if we began including a default jar... in the wheel?

1. and 2. feel like they should be possible to maybe do via a rule, although it might be a pretty hacky rule.

...there might also be solutions with more explicit per-entry dependencies, although i wonder if this magic is essentially unique to pants' own environment because we have so many tests of pants itself. it feels like it.

<@U051221NF> regarding your comment on unit test failures, we can temporarily turn off remote execution for unit tests and they should still pass with local V2 before Travis times out. The more important thing is ITs, because we go from 1 shard to 15 local shards by turning off remoting 

I pretty much understand the problem I think

ok, works for me. i'll take a swing. they shouldn't overlap much.

Hmm we will need to detect inside tests (specifically in `setUp()`) that we are running remote

Is there an env var we can check or some other thing?

Unfortunately we can't access real options inside tests

Not that I know of, but we can update our docker image to set this. In the meantime, update ci.py

I’m past security at airport so can add this now to the Docker image. What should we call the env var, `REMOTE_EXECUTION=true`?

oh wait I can’t do it anymore. I need `<mailto:eric@toolchain.com|eric@toolchain.com>` to be added to the RBE project by the official Pants account