https://pantsbuild.org/ logo
Join Slack
Powered by
Sharing a reflection from recent JS-backend review...
# development
w
Sharing a reflection from recent JS-backend reviews: I think from a usability design perspective 
adhoc_tool
is eating the js backends lunch, which speaks to how versatile 
adhoc_tool
is. Would like to unify the backends so that features added to 
adhoc_tool
naturally fits for js scripts too, i.e 
package_json(scripts=[<adhoc-stuff-like-system_binary-and-workspace_environment-works-here>], ...)
. It looks do-able from a distance but a substantial change.
f
Maybe draft an initial rough design doc?
It would be useful to see what the proposed UX is.
👍 1
w
So fwiw - I use adhoc tool over the js backend as well, and I think it more speaks to how broad and messy that ecosystem is. Many package managers, frameworks, the need to run random scripts (which often reside in package.json), multiple runtimes, workspaces, etc. An example of where this kinda leaks out is that we have 8 (well, really 16 - but 8 due to pluralization) variants just as an equivalent to 
python_sources
and 
python_tests
(due to js, jsx, ts, tsx). I don't really know what the solution is here though - but I think evaluating the api of the js/ts backends would be a start, but to your point - I do think a lot of the internals could benefit from 
adhoc_tool
as an implementation
w
I dont think the number of target types speaks to the tooling space being large or leaking or w/e. It's just a choice 🤷‍♂️ Could as easily have made them all `javascript_source`s, I voted no because source analysis is a key feature of pants and it is hard without target types.
I also dont think the "js landscape is too large and changing" insight helps much, I think we'll just have to accept it as fact and build something that makes it easy to do common things
adhoc tool has some great awnsers to scripting workflows that can be cacheable etc. And 
node_build_script
(and soon to be 
node_run_script
) and 
adhoc_tool
have almost the same api, just one got implemented before the other.
I could be wrong though
for JS only shops I dont think pants has much of place
w
> I dont think the number of target types speaks to the tooling space being large or leaking or w/e. It's just a choice 🤷‍♂️ Could as easily have made them all `javascript_source`s, I voted no because source analysis is a key feature of pants and it is hard without target types. I mean, I think this statement here kinda points to what I was talking about, in that there is currently a bit of an impedance mismatch between JS/TS and Pants. > for JS only shops I dont think pants has much of place 💯
I made this point with a couple of backends (including the Swift one that I keep trying to redesign because it never feels quite right) - where some backends are a better fit for Pants running the show, and others are very much "just let Pants know this exists, and defer as much as possible, as hard as possible"
👍 1
w
I mean, I think this statement here kinda points to what I was talking about, in that there is currently a bit of an impedance mismatch between JS/TS and Pants.
I dont understand this comment, sorry
Where is the mismatch? In that JS have many source types and that means a build file contains many target types?
Or do you mean that there crops up a new one every so often that pants doesnt know about?
I 100% agree about the defer strategy btw
w
Kinda both, to some degree. I recall trying to do something with wasm as well, and it ended up being more complicated than I think it should have been as an already solved problem outside of pants. I'm trying to find some old work I did where I had a mock up of something that I saw as the minimal JS backend, which might be more helpful if I can find it. In short, it was I think 1 or 2 targets, and enumerated scripts into something 
pants run
could use directly. Essentially, the one package.json ran the show, but I think my version assumed that the 
packageManager
field was populated. Then I started moving stuff to deno - and it kinda all fell apart
And similar to the node_build_script kinda thing - you had to essentially "map" your commands into pants language (e.g. "package uses this script" kinda thing)
w
I hear you, and I think 
workspace_environment
could be the "make it easier to build wasm modules" part. But if it is to be sandboxed then you have to provide the sandbox with the stuff, that's the pants model.
💯 1
For the rest (optimizing for minimal build files) I dont know, the source analysis is a key feature we use for compliance tooling at $dayjob, I see no problem with it vs the features it brings
maybe if we drown in special cases on this end
check_node_script
, 
lint_node_script
... etc seem not great
w
Absoutely, I get it. I also just recalled what my original case was (from May 2023): https://gist.github.com/sureshjoshi/98fb09f2a340f7c1dad270c4887865a0 I wanted to build a sveltekit site, but didnt want 137 line long build file to do it 😄 Essentially reduced this to 4 targets: sources, tests, "package", and archive - and maybe a resources/files to pull in the .sveltekit folder
But yeah, my perspective probably just comes from the different use cases I have, which may not overlap with yours as much 🤷
check_node_script
, 
lint_node_script
... etc seem not great
In my case, it was something more like auto-mapping to generate a 
pants run path/to/package:my-script1
and then known goals could be mapped into 
pants lint
- but it was just dictionary-style, so not a lot of boilerplate - but still boilerplate
anyways, probably no resolution here directly - but if you create a rough design doc, I'd be interested in seeing it - as I agree that adhoc tool can be a boon to the js stuff
f
I hear you, and I think 
workspace_environment
could be the "make it easier to build wasm modules" part. But if it is to be sandboxed then you have to provide the sandbox with the stuff, that's the pants model.
A siide note: A backend can run processes in the workspace without needing a 
workspace_environment
provided by a user by directly constructing a 
ProcessExecutionEnvironment
with 
execute_in_workspace=True
and passing via 
implicitly
👍 1
w
Excellent! Likely the design doc will propose a global "run in workspace" option with the explicit goal of avoiding copying 
node_modules
to sandboxes
f
Did you start a draft on the design for this @worried-painter-31382? We've been on the struggle bus trying to get pants and node to cooperate for a bit and would love to contribute back to improving it
f
> Excellent! Likely the design doc will propose a global "run in workspace" option with the explicit goal of avoiding copying 
node_modules
to sandboxes and which can be set up programmatically in rule code without a user needing to define a workspace environment.
w
No sorry, I was at the "grok the code" stage and then reality came back. I recall I thought "migration stages" seemed prudent. 1. Start with adopting the adhoc API for node scripts, still no workspace support, just boilerplate and name changes etc so we dont have to live with both or adapt the JS-backend fields. 2. Change the nodejs rules to be invokable according to @fast-nail-55400 recipe 3. ...
4 Views
Open in Slack
PreviousNext
Powered by