Sorry in advance for raising a potentially thorny ...
# general
Sorry in advance for raising a potentially thorny topic, but we are at a crossroads in continuing our monorepo journey as a substantial portion of apps we want to build will have a React front end. Is there any guidance from the Pants developers on interim tooling for node/js while awaiting the feature in Pants? After a brief survey of the tooling targeting js, it looks like NX with the integrated approach is semantically most similar to Pants: A project.json to define targets/dependencies of a package (analogous to BUILD files), a “factored out” root package.json of third party requirements (analogous to a global resolve), and task executors running over a resolved targets DAG. Bazel with rules_js seems even closer, but not sure it’s worth the learning investment, not to mention it’s not a core feature of Bazel itself.
I wouldn’t discount something for bazel because it’s not part of the core release. Many libraries work in partnership with bazel to ensure compatibility with release cycles. They usually indicate this on their GitHub
👍 1
By the way, at the moment we are building a Python “front end” by using the experimental shell command target to create builds of the React components, this being triggered and the outputs (built css/html/js) copied whenever this new front end (just a simple web server) is packaged/ran. This is quite similar to the package oriented approach in NX, though quite a bit less sophisticated, as the build scripts currently assume a standard package.json setup.
cc @ancient-vegetable-10556 who has been working on simple ways of integrating outside tooling into Pants. So you could e.g., build a react frontend with yarn or whatever and then embed it as a resource into a Python service.
Indeed, that is what we’ve managed to do. But the introduction of custom build scripts, per-project third-party requirements and build configurations, etc. makes me uncomfortable in regards to scale / accruing tech debt if those builds all have to be maintained after Pants comes to support all this. I imagine restricting to this limited use case of typescript where we can provide the pre-defined build scripts and project scaffolding can alleviate those concerns, but what will be next is obviously on my mind, especially as the repo is opened to more developers and people discover and perhaps come to expect they can customize their builds in totally arbitrary ways.
One potential approach could be to use the enhanced
features to build up an initial support base, and then work to factor out some of the more repetitive steps you encounter into plugins
one of the reasons why our JS support is lacking is that there’s been somewhat of a chicken and egg scenario — we don’t have the expertise to build a good JS plugin, since we don’t really spend a lot of time in the ecosystem, but we haven’t been able to attract the expertise because we haven’t supported it at all.
@ancient-vegetable-10556 @happy-kitchen-89482 Just wanted to give a quick update here. We realized there is a fairly generic build pattern for some applications that are primarily Python-based, but depend on a lot of resources being bundled (primarily static Web assets from React and other application frameworks, but also shell scripts, Q scripts, and even compiled binaries). For any such non-Python project that generates resources to be bundled with the main Python project, we have a build script that produces that project's exported resources (e.g., yarn build). This script will also relocate those built resources to the Python project so that people can test their Python applications locally. The basic workflow is: 1. build sub-project assets 2. relocate assets to the appropriate locations in the Python project 3. run, test, etc. Python project. If changes need to be made in the sub-project (e.g., Web front-end modifications), then just repeat (1) and (2) before (3). This is how developers on the repo who are not yet adopting Pants can work. For those adopting Pants, they would define resources referring to (2) and
depending on those
. (Of course this would be done anyway, regardless of the individual developer's use of Pants, because CI, which uses Pants for packaging PEX files, would need this configuration.) Finally, those same
would also depend on a `shell_command`** that invokes the above build script and declares as
the built assets. That build script also checks if the assets were already manually placed (bundled as said resources above) and skip the build script if so. This speeds up local workflows because the build script can take a long time to run and currently there's no caching of the results of
(at least it does not seem that way). If a developer wanted to check the full build and packaging process locally before CI, they could then delete the placed resources or rename them or comment out the dependency in their BUILD file. This redundant dependency means developers have the option of placing built assets directly or allowing Pants to build them at package time. In general, they would place them locally, but not in SCM/CI. * The main reason for sharing this update --* To achieve this, we had to add a new target
when emits
instead of
since Pants won't bundle
. We also implemented
. Technically, the
are what the
are depending on, which in turn depend on the
. I wonder if this should be something Pants officially adds though? Right now it's a local plugin for us, but it seems this would be broadly useful. Finally, could
support caching? I know one of the stated purposes in the documentation is to generate side-effects, but could a provided option indicate that it's for generating resources instead and that it is deterministic (
Shell commands do cache. The “side-effects” you generate are output files.
(same for
and `shell_command`s must be deterministic. Things break if they aren’t.
Finally, to use the output of a
as resources instead of files, try
, available as of 2.16
happy to guide you through any of these if you need it!