hey folks, does pants store external tools somewhe...
# development
c
hey folks, does pants store external tools somewhere? I'm testing that multiple versions of terraform work, and I'm trying to make sure that the correct version of terraform was downloaded
h
it downloads them into
~/.cache/lmdb_store
. But that is an opaque cache that you can't easily inspect
c
oh, I'll just wipe the cache and see how many versions it downloads
h
I don't think it will report it like that. I'm trying to think of the best way to do this. So far, I'm thinking run with
--no-pantsd
so that you don't get in-memory memoization but you still have
--local-cache
enabled. And then somehow block the internet so it would fail to download the tools Or, I think you can inspect
-ldebug
but also I'm not sure this is necessary. Pants's download caching code is already well tested What specifically is your concern?
f
and Pants verifies the SHA-256 digest of the download based on what is in the applicable download option.
c
I'm concerned that I'm doing something silly. I already found myself calling
rule_runner.set_options
twice and overwriting the arg to request a different version.
f
Are you able to share a code snippet?
w
to enable logging in a test, you can also:
Copy code
from pants.testutil import rule_runner
...
@rule_runner.logging
def test_something_something() -> None:
    ...
c
code's [here](https://github.com/pantsbuild/pants/pull/15958) and almost ready for review. I'm pretty convinced it's actually downloading new TF versions. Thanks all for your help!
f
so unless
isolated_local_store=True
is set (defaults to
False
), the test should share the same local store as the
./pants test
invocation. so Terraform downloads should be shared between invocations.
that said, it would be good to verify that is actually happening in your case. https://github.com/pantsbuild/pants/blob/072990760088e21b06835d5ab2b84b6d4de15d44/src/rust/engine/src/nodes.rs#L882 is the actual download code. Maybe add some logging there?
e.g., via
debug!
or
trace!
logging macro invocation
c
ok, I this is my first time with rust, but I think I copypasted well enough. But I only see it printing out when it's fetching the pex binary, even when nuking the cache. I also tried intercepting it higher up, in the python [download_external_tool](https://github.com/pantsbuild/pants/blob/072990760088e21b06835d5ab2b84b6d4de15d44/src/python/pants/core/util_rules/external_tool.py#L316) rule, and it looks like it requests the right url.
f
are you using that
logging
annotation on the test? also is
-ldebug
passed into the
RuleRunner
stuff via
.set_options
?
c
... I have to set the logging level to debug to see the debug logs 🤦. Definitely confirming that the tests are requesting different versions of TF from different URLs. Do we want the debug statements in the rust committed too? Or should I discard them? Also, adding
-ldebug
(to the tested RR or to the top-level pants) didn't seem to make a difference, but I also have
RUST_LOG=trace
in my env so maybe that was doing something too.