wide-midnight-78598
03/13/2023, 6:53 PMadhoc_tool
workflow (https://github.com/pantsbuild/example-adhoc/blob/main/javascript/BUILD), how should a tool like pnpm
be handled, which places all installed deps into a global content store, and then hardlinks them into the local repo? https://pnpm.io/faq#why-does-my-node_modules-folder-use-disk-space-if-packages-are-stored-in-a-global-storeancient-vegetable-10556
03/13/2023, 7:05 PMwide-midnight-78598
03/13/2023, 7:05 PMProcessExecutionFailure: Process 'the `adhoc_tool` at frontend/web:install' failed with exit code 1.
stdout:
Lockfile is up to date, resolution step is skipped
Progress: resolved 1, reused 0, downloaded 0, added 0
Packages: +627
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Packages are copied from the content-addressable store to the virtual store.
Content-addressable store is at: /Users/sj/Library/pnpm/store/v3
Virtual store is at: node_modules/.pnpm
Progress: resolved 627, reused 112, downloaded 0, added 101
Progress: resolved 627, reused 230, downloaded 0, added 231
Progress: resolved 627, reused 391, downloaded 0, added 390
Progress: resolved 627, reused 411, downloaded 0, added 409
Progress: resolved 627, reused 477, downloaded 0, added 471
Progress: resolved 627, reused 564, downloaded 0, added 560
.../node_modules/@sveltejs/kit postinstall$ node postinstall.js
ELIFECYCLE Command failed with exit code -2.
Progress: resolved 627, reused 626, downloaded 0, added 627, done
.../node_modules/@sveltejs/kit postinstall: Failed
However, when I open the sandbox and run it manually:
⏺ T/pants-sandbox-y2xLyl % /bin/bash -c $'cd frontend/web && /usr/local/bin/pnpm install' ⎇
Lockfile is up to date, resolution step is skipped
Packages: +627
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Progress: resolved 627, reused 626, downloaded 0, added 0, done
node_modules/.pnpm/@sveltejs+kit@1.10.0_svelte@3.55.1+vite@4.1.4/node_modules/@sveltejs/kit: Running postinstall script, done in 1s
Done in 3.4s
ancient-vegetable-10556
03/13/2023, 7:07 PMwide-midnight-78598
03/13/2023, 7:07 PMancient-vegetable-10556
03/13/2023, 7:08 PMnpm
iswide-midnight-78598
03/13/2023, 7:09 PMstderr:
npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility>
npm WARN deprecated @types/sass@1.45.0: This is a stub types definition. sass provides its own type definitions, so you do not need this installed.
npm ERR! code ENOENT
npm ERR! syscall spawn sh
npm ERR! path /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-96EvqO/frontend/web/node_modules/@sveltejs/kit
npm ERR! errno -2
npm ERR! enoent spawn sh ENOENT
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/sj/.npm/_logs/2023-03-13T19_08_14_541Z-debug-0.log
ancient-vegetable-10556
03/13/2023, 7:11 PMnpm ERR! syscall spawn sh
what’s this about?wide-midnight-78598
03/13/2023, 7:14 PM"postinstall": "node postinstall.js"
ancient-vegetable-10556
03/13/2023, 7:15 PMPATH
.node poastinstall.js
suggests to me that it might be attempting to spawn a shellwide-midnight-78598
03/13/2023, 7:16 PMsh
which doesn't make sense to meancient-vegetable-10556
03/13/2023, 7:16 PMadhoc_tool
only has things in the PATH
that you say need to be therewide-midnight-78598
03/13/2023, 7:16 PMancient-vegetable-10556
03/13/2023, 7:16 PMsystem_binary
for sh
, and adding it as a runnable_dependency
on the adhoc_tool
wide-midnight-78598
03/13/2023, 7:17 PMancient-vegetable-10556
03/13/2023, 7:17 PMwide-midnight-78598
03/13/2023, 7:18 PMancient-vegetable-10556
03/13/2023, 7:18 PMwide-midnight-78598
03/13/2023, 7:20 PMsh
- 🎉ancient-vegetable-10556
03/13/2023, 7:21 PMpnpm
not use the global store, otherwise you are going to get weird cacheability resultswide-midnight-78598
03/13/2023, 7:22 PMancient-vegetable-10556
03/13/2023, 7:22 PMwide-midnight-78598
03/13/2023, 7:23 PMancient-vegetable-10556
03/13/2023, 7:24 PMadhoc_tool
I can sit in a corner, visibly sobbing, saying “please don’t do it that way” while the world ignores me 😄{chroot}
wide-midnight-78598
03/13/2023, 7:25 PMworkdir
?ancient-vegetable-10556
03/13/2023, 7:26 PMf"{build_file_dir()}/blah
if you really need it.wide-midnight-78598
03/13/2023, 7:28 PMnpm ERR! enoent ENOENT: no such file or directory, open '/Users/sj/Developer/mydir/package.json'
Where the BUILD is located in /Users/sj/Developer/mydir/frontend/web
run_shell_command(
name="build",
command="npm run build",
execution_dependencies=[":install",],
)
I would have thought it was relative to the BUILD rootancient-vegetable-10556
03/13/2023, 7:28 PMrun_shell_command
is not executed in the sandboxwide-midnight-78598
03/13/2023, 7:29 PMpants
was executed? Or project root?ancient-vegetable-10556
03/13/2023, 7:29 PMcd {chroot} &&
in front of the command; or move files into a useful locationwide-midnight-78598
03/13/2023, 7:31 PMancient-vegetable-10556
03/13/2023, 7:31 PMwide-midnight-78598
03/13/2023, 8:02 PMRun npm run preview to preview your production build locally.
> Using @sveltejs/adapter-static
Wrote site to "build"
✔ done
ancient-vegetable-10556
03/13/2023, 8:02 PMwide-midnight-78598
03/13/2023, 8:04 PMrun_shell_command
doesn't cache, but the adhoc_tool
does, correct?ancient-vegetable-10556
03/13/2023, 8:04 PMwide-midnight-78598
03/13/2023, 8:04 PMancient-vegetable-10556
03/13/2023, 8:05 PMexecution_dependencies
vs output_dependencies
thoughwide-midnight-78598
03/13/2023, 8:21 PMadhoc_tool
and run_shell_command
to build a Typescript + SvelteKit example (basically, npm install
and then call package.json commands)
# Grabbing build/bundle-centric files, as well as eslint/prettier files
files(
name="build-meta",
sources=[
"*eslint*",
"*prettier*",
"*npm*",
"*.json",
"*.cjs",
"*.js",
"*.ts",
"*.yaml",
]
)
# The source files themselves - TODO: exclude the *.test.ts files
# Make sure to grab `svelte-kit` which is part of a post-install
files(
name="sources",
sources=[
".svelte-kit/**/*",
"src/**/*",
"static/**/*",
]
)
# Node/SvelteKit's post-install needs the host's `sh` to be in the environment
system_binary(
name="sh",
binary_name="sh",
fingerprint_args=["--version"],
)
# Pull the host's `node` into the environment
system_binary(
name="node",
binary_name="node",
fingerprint=r"v(19|[2-9][0-9])\..*\..*",
fingerprint_args=["--version"],
)
# npm, yarn, etc... The tool should attempt to install to a local folder for better caching (i.e. not a global store ala pnpm?)
system_binary(
name="npm",
binary_name="npm",
fingerprint_args=["--version"],
fingerprint_dependencies=[":node"],
)
# Perform and cache the `npm install` step, `:build-meta` is required for this step, but the `:source` should be passed through
adhoc_tool(
name="install",
runnable=":npm",
runnable_dependencies=[":sh", ":node"],
args=["install"],
output_dependencies=[":build-meta", ":sources"],
execution_dependencies=[":build-meta"],
output_directories=["node_modules"],
timeout=300,
)
# Run build directly in the appropriate working directory - this should be an `adhoc_tool` if it
# will be output to the local working directory
run_shell_command(
name="build",
command="cd {chroot}/frontend/web && npm run build",
execution_dependencies=[":install",],
)
# Try running my tests
run_shell_command(
name="test",
command="cd {chroot}/frontend/web && npm run test",
execution_dependencies=[":install",],
)
broad-processor-92400
03/13/2023, 9:19 PMshell_command
/ adhoc_tool
invocation to actually do the build, and then run_shell_command
that does whatever non-cached follow-up is required (e.g. copying things back to the repo, if integrating with existing dev workflows)wide-midnight-78598
03/13/2023, 9:27 PMadhoc_tool
for the build process - and that makes much more sense.
The next thing to see if if I can use this for setting up my ansible deployment to be a bit cleaner. There is a plugin in the works, but there are so many usecases to think about - that right now, starting with this set of commands for my immediate deployment is great.ancient-vegetable-10556
03/13/2023, 9:28 PMlint
and check
yet. I had a half-baked design that I might get to later this month, or I can talk it through with someone.wide-midnight-78598
03/13/2023, 9:28 PMrun
with a fail is all it needs to do todayancient-vegetable-10556
03/13/2023, 9:28 PMexperimental_test_shell_command
, but that has not kept up with `shell_command`/`adhoc_tool` featuresrun
will work as a replacement for pass/fail type arrangements, but if you try to modify code, there’ll be dragonsfix
-type workflows won’t work)wide-midnight-78598
03/13/2023, 9:29 PMfmt
pluginancient-vegetable-10556
03/13/2023, 9:29 PMrun_shell_command
to be sufficient for your needs, or would you prefer something that matches adhoc_tool
a bit better?happy-kitchen-89482
03/13/2023, 9:32 PMfiles()
target.broad-processor-92400
03/13/2023, 9:36 PMare you findingFor most of them, we'd like something like https://github.com/pantsbuild/pants/discussions/18235, because we're basically doing codegen that gets checked-in, and having pants manage that natively would be much better than having to remember to manually run each of the codegen commands in sequence, as appropriate. For the other one, we're runningto be sufficient for your needs, or would you prefer something that matchesrun_shell_command
a bit better?adhoc_tool
open
on a docs HTML file (after copying to a fixed location to avoid a proliferation of browser tabs), so run_shell_command
is fine for that.