https://pantsbuild.org/ logo
Join Slack
Powered by
Is there anything trivial on 2.17 that needs to be...
# development
a
Is there anything trivial on 2.17 that needs to be done to make the new JS backend not throw errors when doing `--changed-since`/anything else that scans the repo?
b
Does it require some tool locally? 
go
backend had/has this problem šŸ˜ž I just disable it in my local pants config for the repo
a
canā€™t, there are 
javascript_sources
in BUILD files
b
šŸ« 
a
If itā€™s just an environment thing on my machine, I can fix it, otherwise, Iā€™d rather see the changes backed out until Pants works cleanly with the backend enabled
@worried-painter-31382 ^ping
w
What's the error sorry?
a
Filing a ticket, and Iā€™ll link momentarily
šŸ‘ 1
w
stu found weirdness w.r.t there being a package.json file in some directory I cannot see (I think it was in the doc directory?), I have https://github.com/pantsbuild/pants/pull/18523 for that particular error
a
w
hmm
a
Happy to debug whatever is necessary on my machine, but generally speaking Iā€™d prefer to see the backend be able to be turned off if our general development instructions canā€™t be followed
w
There's very little that can actually go wrong here (in theory), I'll have to check what the silicon tarballs contain for the node install
a
let me know if I can look at anything for you; my strong preference is that this works šŸ™‚
w
The tarball looks the same to me as intel linux, except the exe (for obvious reasons...) Can you preserve the sandbox? Is the file (
../lib/cli.js
) in there? Do you have read access in those folders? Have you brew installed npm or node (should not be required, just wondering because I notice that for some reason 
/bin
is set in the process PATH)?
a
OK, working from backwards: I have npm and node on my machine
letā€™s see about preserving the sandbox
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants-sandbox-03GF3M % ls -la
total 72
drwxr-xr-x    8 chrisjrn  staff    256 Mar 27 11:28 .
drwx------@ 508 chrisjrn  staff  16256 Mar 27 11:28 ..
lrwxr-xr-x    1 chrisjrn  staff     45 Mar 27 11:28 ._npm -> /Users/chrisjrn/.cache/pants/named_caches/npm
lrwxr-xr-x    1 chrisjrn  staff    155 Mar 27 11:28 __node -> /private/var/folders/0m/h2n902qn38b8555xq8z0w_h40000gn/T/immutable_inputs9SG4Ex/.tmpXga02S/a97a3f6ba08afdca934b08117b3f19f04975937860fa883bf39cc26c20ec8406
-rwxr-xr-x    1 chrisjrn  staff    357 Mar 27 11:28 __run.sh
-rw-r--r--    1 chrisjrn  staff  23261 Mar 27 11:28 package-lock.json
-rw-r--r--    1 chrisjrn  staff    528 Mar 27 11:28 package.json
-rw-r--r--    1 chrisjrn  staff   1452 Mar 27 11:28 script.cjs
w
right, so the 
__node
is the place where I'd expect the hermetic node installation be
and for the 
lib/cli.js
(it's an node/npm internal script) to be
Would be interested to see what 
node
your 
__run.sh
eventually picks up. I'm not very good with mac but I see at least 
/usr/bin/env bash
and 
usr/bin/env node
shebangs in the code path to where you hit the error, so maybe the other installs are polluting the sandbox somehow? Still doesn't really explain why a node install is missing lib/cli.js, but maybe that is just a symptom of something else?
a
Yeah, if youā€™re seeing calls to 
env
, things are probably going to go wrong šŸ™ƒ
unless youā€™re being very specific about constructing $PATH
w
I guess adding 
/bin
to it isn't considered specific šŸ˜…
a
OHGODNO
w
hehe, that was heritage, I didn't dare touch that when I started this JS journey
a
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants-sandbox-03GF3M % which node
/opt/homebrew/bin/node
chrisjrn@chrisjrns-MacBook-Pro pants-sandbox-03GF3M % /opt/homebrew/bin/node --version
v19.7.0
so thereā€™s only one 
node
on my system
where are you constructing those processes?
Letā€™s take a look at how the process environment is constructed
w
node is installed via a binary download and sent into the process via an immutable digest
then it is exposed by adding that installed-to-dir to PATH
a
where in the codebase?
w
src/python/pants/backend/javascript/subsystems/nodejs.py:237 for the env
a
OK, let me take a look
OK, well 
/bin/bash
is ancient, and 
/usr/bin/bash
is not.
So 
PATH
is just 
/bin
and wherever the self-contained SDK manifests?
w
yarp
a
Iā€™d probably want to use 
BinaryShimsRequest
to get 
bash
etc on the 
PATH
rather than using 
/bin
šŸ‘ 1
Letā€™s see if the error goes away if it calls a modern 
bash
w
I think you need 
sh
as well, just a heads up
a
Copy code
"PATH": f"/usr/bin:/bin:{self.binary_directory}",
doesnā€™t change the output afaict
šŸ˜• 1
w
Another thing to test I guess is setting 
PANTS_NODEJS_KNOWN_VERSIONS
to nothing and set the version to your local install, it should find any node you have on PATH, which I guess should work if the hypothesis is what I think it is
Wouldn't really tell us much about why the downloaded install is being prickly though...
a
Copy code
version, platform, sha256, filesize = [x.strip() for x in version_str.split(ā€œ|ā€)]
ValueError: not enough values to unpack (expected 4, got 1)
hang on
overriding 
PANTS_NODEJS_VERSION=19.7.0
worked
w
ah, right
PANTS_NODEJS_KNOWN_VERSIONS
wouldn't match that, of course
(I think the other error was an empty string as list-only element instead of empty list thing)
a
aha
w
Soo when pointing to the brew installed node it "all works"? Or just the option parsing?
a
The rules that were getting picked up did not fail
so it eventually got through to running the tests I wanted to run
I note that our CI isnā€™t running against mac os 
arm64
, perhaps (annoyingly) this is relevant
w
Mm, I just tried installing npm via 
apt
(ubuntu 22), which I guess is the closest equivalent we have here, and it is still well behaved on 
main
running
Copy code
./pants --no-local-cache test src/python/pants/backend/javascript::
which npm
/usr/bin/npm
a
let me check that invocation on my machine
that exact invocation passes
w
oh
a
Now checking --changed-since with local cache switched off
(so slow)
that also passes
and switching the local cache back on fails
well thatā€™s just weird
so that leads me to believe this is a caching bug on my machine. Letā€™s see what happens if I move the pants cache out of the way for a moment
w
the tar contains symlinks, is there a gotcha there with immutable digests? I'm well out of my depth on pants intrinsics šŸ˜¬
a
The experts on symlinks are probably @bitter-ability-32190 and @witty-crayon-22786
(that said, blowing away my pants cache and starting again seems to have fixed the issue)
w
Im just postulating based on the error, it would makes sense that a symlink that somehow got treated as a file would not find relative paths
The 
BinaryShim
comment is probably still relevant though I assume? Should I create an issue for that?
w
symlinks should be correctly handled when a tar file is extracted/captured/splatted later
a
@worried-painter-31382
The 
BinaryShim
comment is probably still relevant though I assume? Should I create an issue for that? (edited)
Yeah, you should be explicitly putting binary dependencies on your path (possibly with version-matching tests if any binaries are particularly version-sensitive; POSIX tools arenā€™t)
w
https://github.com/pantsbuild/pants/issues/18609 please add the javascript label šŸ™‚
Open in Slack
PreviousNext
Powered by