https://pantsbuild.org/ logo
Join Slack
Powered by
Am I crazy? ```$ pants package apps/app1:docker 2...
# general
g
Am I crazy?
Copy code
$ pants package apps/app1:docker
22:05:53.95 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ValueError: Could not find a binary with name `pulumi` with output matching `^v3\.` when run with arguments `version`. The following paths were searched: /usr/bin, /bin, /usr/local/bin, /opt/homebrew/bin.


$ /usr/local/bin/pulumi version
v3.108.1
b
Not sure! What's the 
BUILD
file(s) you're working with?
g
Copy code
system_binary(
    name="pulumi",
    binary_name="pulumi",
    fingerprint=r"^v3\.",
    fingerprint_args=["version"],
)
This fixed the issue, but based on the error message I wouldn't think I need to add 
/usr/local/bin
Copy code
diff --git i/apps/app1/BUILD.pants w/apps/app1/BUILD.pants
index 8356b9bb20..33de01d8bb 100644
--- i/apps/app1/BUILD.pants
+++ w/apps/app1/BUILD.pants
@@ -84,6 +84,7 @@ pex_binary(
 system_binary(
     name="pulumi",
     binary_name="pulumi",
+    extra_search_paths=["/usr/local/bin"],
     fingerprint=r"^v3\.",
     fingerprint_args=["version"],
 )
b
yes, that does seem unexpected. Can you file an issue?
✅ 1
g
Should using "<PATH>" here work? I didn't find it documented but found it in the source code, but I'm not exactly sure if it should. I tested and it didn't expand it.
h
Hmm, I don't recall offhand, but since you tested it and it didn't expand it, then I guess not? Perhaps it should though
You did find it in the source code for 
system_binary
? It's true that we expand 
<PATH>
in various path-related options, but I'm not sure about target fields like 
extra_search_paths
g
no, here: src/python/pants/core/util_rules/system_binaries.py
h
Ah yes, so we expand it for the option, but presumably not for the field
However I'm not sure that would solve the issue anyway
since 
/usr/local/bin
is in the defaults anyway
Something fishy is going on here
Oh, one thing to try: replace that 
^v3\.
with an empty string, temporarily
Let's see if the problem is "can't find a binary to run" or "when I run the binary I don't get output that matches that pattern"
g
@happy-kitchen-89482 good idea. I'll try that.
yep, that fixed it.
I'm removing the remote cache to be sure though.
no, it definitely fixed it. So it looks like it's not a PATH issue, but a fingerprint problem. It's super weird that setting extra_search_paths fixes the problem.
It turns out it was some weird caching issue.
As soon as I ran a 
git clean -fdx
and 
rm -rf ~/.cache/pants
-- it worked
h
Hmmmmm
OK, so that at least makes sense
so now it works even with the 
^v3\.
match?
👍 1
OK, so there is still a caching issue of some kind, but absent a repro that will be hard to look into...
g
I can try to repro
OK, easy to repo. Just run pants with the binary that is referenced in the system_binary missing, completely. Then install the binary and re-run. It continues to fail over and over again.
It seems like removing .pants.d/ is enough, not the cache.
h
What about restarting pantsd?
(use 
ps -ef | grep pantsd
to find the binary pid and kill it)
I think that should suffice
g
I'll try that.
b
Ah, can we mark those "find a binary" searches as non-cacheable on failure?
h
We can, although the opposite problem will still hold - what if the binary passes and then we uninstall it...
b
Ah, hm. I'd guess the "fail due to missing then install" direction is more common, and potentially more annoying if results are cached. It looks like this might be explicit behaviour https://github.com/pantsbuild/pants/blob/908d24562c00744369df88658d55ac884cba78b9/src/python/pants/core/util_rules/system_binaries.py#L573-L575 ?
Copy code
# NB: Since a failure is a valid result for this script, we always cache it,
                # regardless of success or failure.
                cache_scope=env_target.executable_search_path_cache_scope(cache_failures=True),
The environment changing seems to be discussed in 
executable_search_path_cache_scope
, and it looks like it's cached per-pantsd for local invocations.
g
Honestly I think the current behavior is fine. Just adding some sort of information to the error message would be helpful to short circuit development of anything that depends on a system_binary.
c
h
@gentle-flower-25372 a PR improving the error message would be welcome! Have you confirmed that killing pantsd is sufficient?
9 Views
Open in Slack
PreviousNext
Powered by