Hey everyone, I’m a founder at Trinio, a startup from Brazil working on a fast checkout for e-commerces. Currently, our codebase is all in Node and we had been using turborepo to manage our monorepo. We’re slowly switching some services to Golang as we migrate them to AWS lambdas, which prompted the need for a new tool. Since the engineering team is pretty heads down working on the product, I volunteered as a tribute to run some tests with some build tools. At this point I tried Bazel, Please, both of which had a few dealbreakers for us, and now Pants, which seems promising so far. I echo Tony in that I’d love to contribute at some point too!

Haha, love a good Hunger Games reference. Do hit us up with as many skeptical questions as you need to feel confident that you've finally landed on a good fit. If it's not a good fit, we'll definitely tell you. We'd rather that devs get their needs met well, regardless of whether it's by Pants or something else. By the way, have you seen @worried-painter-31382's ongoing work on JS/TS support? I know he'd really appreciate feedback from organizations that would be likely to use it, and guidance on which choices would make it most useful. So you've arrived at a great time to have impact right away, just by giving thoughtful feedback.

Totally have! I’ve seen the effort on Github and checked all opened issues on the matter. That was what made me decide to try it. For now I’ve been implementing using the new

adhoc_tool

but the plan was to switch to the JS/TS native backend once that’s ready. In particular we need

pnpm

or

yarn

support since we’re heavily relying on workspaces.

Ah cool! You're way ahead of me. Excellent. I know he wants to make sure that effort can prove useful for more people than just himself, so communicating about what you'd find most useful would be very appreciated and would probably help move things forward more.

Sounds good! @worried-painter-31382 hit me up whenever, glad to share what my ideal pants usage would look like!

In the meantime, any feedback you can offer on those issues would probably be useful if you've got time to try stuff out and or suggest approaches.

Happy to try stuff out as soon as pnpm or yarn works. I can’t really have a real life use case for the tool with plain npm. AFAIK that development is still in progress, right?

pnpm support pr will probably be open this weekend :)

I love when someone asks a question at the perfect moment like that.

Amazing, I’ll keep an eye on it and test as soon as it’s open!

💃

https://github.com/pantsbuild/pants/pull/18864 There it is 🙂

Awesome, I’ll give it a try! @worried-painter-31382 is there an easy way to test this PR with scie-pants or should I just clone the repo and pull the branch?

Sorry, I was on release duty (not for pants, though) 🙂 Thank you for testing, I think there's a lot of rough edges needs ironing out, but hard to find them without getting something out the door!

Hey @worried-painter-31382, pardon me if I end up asking stupid questions! I’m going through the code to try to understand what are the targets and goals that you implemented and how to use them but since I’m new to the code base I might be understanding things wrong. I added a

package_json

target to the root of our pnpm workspace folder and to start testing things out just ran

pants package //node:package

. It starts installing node modules in the sandbox but then I get an error:

Failed to digest inputs: "Failed to open \"/private/var/folders/rz/n_vkc61s0_90pc18wy28l4mw0000gn/T/pants-sandbox-9xQzu3/node/node_modules/.pnpm/@typescript-eslint+typescript-estree@5.56.0_typesc
ript@4.9.3/node_modules/@typescript-eslint/typescript-estree/dist/node-utils.js\": Too many open files (os error 24)"

I tried setting a higher ulimit with

ulimit -n 100000

but still got the same error.

So

pants package <package_json_target>

will produce a an output the same as

pnpm pack

would do. If you're looking to produce a transpiled output, a

node_build_script

is probably more similar to what you'd do in turborepo or similar

Gotcha! At first I thought that node_build_script was a target. Is there a way I can set system binaries that need to exist for the build script to run? In my case, we’re building a Next.js application and I’m getting two errors at the moment. 1.

uname: command not found

- I believe probably bc PATH hasn’t been set in the sandbox so Next can’t find uname. 2.

Couldn't find a pages directory

. I believe probably bc I haven’t specified what are the sources that should be moved to the sandbox. Tried settings the

_sources

argument in

node_build_script

but something tells me that this looks like a private field, lol

Hmm, indeed node_build_script:s are not a target per se, more similar to

adhoc_tool

if you're familiar. 1. No, not yet. I want to introduce feature parity with

adhoc_tool

here, since it seems nodejs ecosystem has some amount of assumptions about what binaries that are ~always available. You probably can work around it by assigning

PATH=

in the build script 2.

dependencies

yes! If you have javascript sources, pants can infer this for you, if you dont, you'd need a `resources`target and set the dependencies in the

BUILD

file

Got it! They might not work for now then since it’s mostly tsx files. I had to add the other workspaces that this package depends on as well. For instance, we have a

types

lib workspace that this project imports and it’s marked as

workspace:^types

in

package.json

. Pants couldn’t infer that dependency right away (maybe because it’s also a TS project?), so I had to add it to

dependencies

Did you map the other lib to a

package_json

?

Not yet!

That explains why pants couldn't find it

lol… yep! 😄

I think I agree, I was just hesitant to make it optional because npm does not have workspace protocol support (but does support workspaces). Omitting versions have some side effects, pants-codewise. Feel free to open an issue for it.

Just mapped the other libs to a package_json but the source files aren’t being copied to the sandbox

Which sandbox?

pkg1

depends on the

types

source files (not transpiled).

types

are in

pkg1

package.json dependencies as

workspace:^types

. I added

package_json

to the

types

BUILD file as you mentioned before to see if pants would infer the dependency and copy the source files but it didn’t. So I added the source files to

package_json

in the

dependencies

parameter. No success as well.

Hmm, the last bit surprises me.

If I explicitly add

types:files

to the

pkg1

dependencies it works. What I can’t get working is the implicit resolution of dependencies in the workspace

What does

pants peek <path/to/types/package.json>

say?

It only shows the third party dependencies listed in

package.json

. The dependencies I set up explicitly also don’t show up, FIY

{
  "dependencies": {
    "types": "workspace:^"
  }
}

This is the dependencies in pkg1?

Yep!

Ok, just checking because if it was "workspace:^types" or some other form it will not work (it being inference)

Yeah, that was a typo on my side, sorry!

That is interesting, if you run

pants list

you'll get a list of targets under a directory

List shows the files

node/libs/types#__techdocs/cli
node/libs/types#__trinio/types
node/libs/types#jest
node/libs/types#node/libs/types/package.json
node/libs/types#ts-jest
node/libs/types#tslib
node/libs/types#typescript
node/libs/types#vite
node/libs/types#vite-dts
node/libs/types#yarn.lock
node/libs/types/.eslintrc.json:files
node/libs/types/BUILD:files
node/libs/types/README.md:files
node/libs/types/jest.config.js:files
node/libs/types/package.json:files
node/libs/types/tsconfig.json:files
node/libs/types/tsconfig.lib.json:files
node/libs/types/tsconfig.spec.json:files

Is your org called @trinio? 🙂 I think I might know what is failing

It is!

The inference isn't smart enough to elide the @org part of the package names

Just so you know, I’m not omitting in my files. I was just omitting them here on slack to be more succint

Ah, swing and a miss

Meaning the package.json still lists

{
  "dependencies": {
    "@trinio/types": "workspace:^"
  }
}

Hmm, and to double check, your

BUILD

for the

types

package is something like

package_json(dependencies=[":sources"])
resources(name="sources", sources="*.tsx")

and

pkg1

package_json(dependencies=[":sources", "lib/types:sources" , "lib/types/package.json"])
resources(name="sources", sources="*.tsx")

?

Yes, but instead of using

resources

I was using

files

is that a problem?

No, should not matter

Not really. It only works if I add

package_json(dependencies=["//lib/types:sources"])

to

pkg1

Right, but essentially the only way it will work is if the package depends on the sources in the dependency, not the package itself

Ah, makes sense!

Yes, because then pants could see that e.g

lib/pkg1/file.tsx

has

import { MyType } from "types/MyType"

, pants will know

lib/types/MyTypes.tsx

is a dep, pants will see that

lib/types/package.json

is the owner, it will be installed, all is well

@worried-painter-31382 If I want to package a docker_image using the

package_json

output, theoretically I just need to add

:package-json

as a dependency, correct?

Mm, docker integration is not something I've had time to work on. What usecase does your org have for docker + nodejs? Do you use it for the runtime, building frontend, both?

@curved-television-6568 created Pants' docker support and might have some insight to lend here too.

I’m not sure where this

recursive

option is coming from…

It seems to be a pnpm error. When I run

pants package //node:docker

it is first trying to package the

package_json

dependency. For that it runs

pnpm pack

and apparently

recursive

is being passed as an option, which is invalid.

I think it is a side effect of your dependency graph stating

docker_image

depends on the

package_json

. I think what you want to do is be more specific in your dependencies here. Is pants in charge of the package:able artifact that will run the server?

Is pants in charge of the package:able artifact that will run the server?

That’s what I was hoping to do.

I think it is a side effect of your dependency graph stating

docker_image

depends on the

package_json

. I think what you want to do is be more specific in your dependencies here.

Is there a way to be more specific here? I tried depending on

package-json#build

but it still asked to package.

Is there a way to be more specific here? I tried depending on

package-json#build

but it still asked to package.

I dont have the code in front of me, but it could be that there's a sneaky dependency between

package_json#build

and the

node_package

target (which is what is "packed"). You can use the

!!

operator to force the target out of your dependencies in the

docker_image

I would consider this a bug if that is the case, though

I can’t find the node_package target. Is it supposed to be “generated” (sorry, don’t know the name of these # “targets”) by

package_json

as well?

Indeed, it is a generated target. I would expect it to have name akin to

__trinio

(target names are not allowed to have the @ character)

The name of the package is @trinio/checkout. When I run

pants list

I can see it as

package-json#__trinio/checkout

is that the one you’re referring to that I should remove with

!!

as a dependency?

I think so

If I deny that dependency it doesn’t copy the built files to the docker_image sandbox 😞

In general I think docker_image only works with packageable targets and

file

, so the

#build

target needs to be bundled into a packageable target (an archive, a pex_binary or similar) for it to eventually show up in the docker build context, maybe you already have that set up, but just fyi

With

adhoc_tool

I was using it directly. With

package_json

I was hoping I could use directly too. I tried adding an intermediary

files

target that would collect the output from

package-json#build

and then use that as a dependency to

docker_image

but it still tries packaging

@trinio/checkout

.

adhoc_tool

produces

file

,

node_build_script

produces

resource

because a likely envisioned scenario was to produce some fronted bundle via the js backend and package it in a pex.

I tried adding an intermediary

files

target that would collect the output from

package-json#build

I don't think that is a possibility with pants?

Would you mind creating a minimal reproduction so that I can write an issue for this?

Sure, I’ll work on it now. I think this is a pretty common use case, tbh. People will either want to build a docker image with the output or package it to deploy to an AWS Lambda function, Heroku or whatever.

The intention is for

pants package

targeting a

package_json

to produce a npm(-like) distributable package (think library authors). The closest equivalent is the python

python_distribution

target. There is no equivalent to the

pex_binary

in the nodejs backend yet, but I guess that could be a way forward. . The

node_build_script

producing

resource

and only

resource

can be considered MVP-pains. It is part of the inital "wants" of the js backend, that a

pex_binary

can depend on a js target that produces

resource

s. Your

docker_image

target triggering

pnpm pack

isn't expected. As you noticed the export-codgen goal did not run

pnpm pack

. This is some side effect of the inferred dependencies of the backend which might be broken, I don't know. I am however not surprised the current docker backend didnt "just work" at any rate, it's current implementation is... limited. It works well with

pex_binary

and

deploy_jar

(and

adhoc_tool

, I guess? haven't tried.)

Yeah, it makes sense!

Im going to spend some devtime on this this weekend, might have better awnsers after. node_test_script only works together with javascript_test, presently

@worried-painter-31382 thanks for following up! Seems like these are great solutions to the issues and use cases I tested. As for the

pnpm pack --filter

issue , as a suggestion you could

cd

into the workspace folder (or use it as a working dir) and run

pnpm pack

without

--filter

. This seems to work and would solve that issue for when people are using

npm_distribution

. Excited to test these solutions when they get merged!

Not that I know of, short of maybe "tricking" pants into treating typescript files as javascript