does anyone know if there any plugins for building webpack bundles?

Not that I'm aware of. I really want to add support for running ESLint and Prettier though: https://pantsbuild.slack.com/archives/C046T6T9U/p1629170996254000 I'm still playing with how to model that, but I think likely I'd expect

npm

to be already installed similar to how Python interpreter selection works. And then for each tool, have a dedicated

package.json

and corresponding lockfile. The

package.json

would be created behind-the-scenes by reading the option

[prettier].version

etc. The lockfile should probably be checked-in on disk, similar to what we're doing for Python "tool lockfiles" like

[black].lockfile

I also want to investigate using

yarn

instead of

npm

sorry how you would create the individiual package.jsons could you elaborate on that a little bit?

Ah sorry, I realize now that was kind of brain-dumpy and not very clear. Taking a step back, I was describing my thinking for how Pants can support installing JS tools like Prettier or Webpack. That's only one piece of the equation for how Webpack support would work. Imo, we want to give the user a nice, declarative API through the options system, where you can set things like

[prettier].version

and

[webpack].version

in

pants.toml

. We also want to make sure installing tools is determistic, so we need to support using the lockfiles from NPM/Yarn, and those should be saved on disk and checked into VCS. The individual

package.json

piece is that Pants will convert your options like

[prettier].version

into a

package.json

for you, without it being saved to disk. It happens behind-the-scenes. But the user gets back a way to run tools like Prettier/Webpack, along with a lockfile they can check in. Does that make sense?

ah that makes a lot more sense - kind of like how i am generating the Dockerfile how are you thinking users would add 3rd party dependencies to their code if they aren’t supplying the package.json? (It is very possible I am still missing the point)

Ah so that's a part I didn't explain, great question. Difference between "tool requirements" vs "user requirements" . Tool requirements (~"dev dependencies") are only for the tools you run, like Jest and Prettier. User requirements are for your own code that you import. With user requirements, you indeed would provide your own `package.json`(s) But for tools, we want to allow you to install those more easily and without coupling to your own user requirements + other tools. Installing Prettier should ideally not be coupled to installing ESLint, which happens usually in JS projects because they all get smashed together into the "dev-dependencies". This is like how in Python w/o Pants, you usually use a single venv for the whole project. -- It is plausible this separation of "tool requirements" and "user requirements" doesn't map well to JS. In Python, this modeling is working really well for us, but JS ecosystem may be different -- Webpack is tricky because you need not only the tool requirement, but also user requirements. Whereas Prettier doesn't need to worry about user requirements, and I think ESLint doesn't either but still need to verify

mmmm yeah that makes sense thanks! Depending on our priorities I might continue this line of questioning if that’s ok!

Absolutely! I'd love for us to have JS support, even if it's not every feature. And regardless of if you have the time to implement, it's super helpful to talk this all through With the Pants engine, the hard part is often design, rather than implementation. Once you have the design + are familiar with the Rules API, it becomes fairly mechanical to implement

I’m looking into this and one problem has come up immediately - I’m trying to call

npm install

to install webpack however I am getting the error:

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'installing webpack' failed with exit code 127.
stdout:

stderr:
env: node: No such file or directory

it looks like node needs to be in the environment cause npm shells out to it. Is there a good way to execute npm with node in the environment?

How are you calling out to

node

? Is it on your

$PATH

?

@happy-kitchen-89482 here is the code - I’m verifying that node is in the PATH of the code I am calling but I think it is not included in the process context (Because I am calling npm and not node directly)

<http://logger.info|logger.info>("Getting npm")
    nvm_bin = await Get(Environment, EnvironmentRequest(["NVM_BIN"]))
    <http://logger.info|logger.info>(nvm_bin)
    search_path = ["/bin", "/usr/bin", "/usr/local/bin"]
    if nvm_bin:
        search_path = [nvm_bin.get('NVM_BIN')]
    npm_path, node_path = await MultiGet(
        Get(
            BinaryPaths,
            BinaryPathRequest(
                binary_name="npm",
                search_path=search_path,
            ),
        ),
        Get(
            BinaryPaths,
            BinaryPathRequest(
                binary_name="node",
                search_path=search_path,
            ),
        ),
    )
    
    <http://logger.info|logger.info>(npm_path)
    process_path = npm_path.first_path.path
    <http://logger.info|logger.info>(process_path)
    <http://logger.info|logger.info>(node_path.first_path.path)
    # PATH = node_path.first_path.path
    # node_binary_location = await Get(Snapshot, PathGlobs([" ../../.nvm/versions/node/v10.15.1/bin/*"]))
    # <http://logger.info|logger.info>(PATH)
    # <http://logger.info|logger.info>(node_binary_location.files)
    
    
    process_result = await Get(
        ProcessResult,
        Process(
            argv=[process_path, "install", "webpack", "webpack-cli", "--save"],
            env={"PATH":tuple(search_path)},
            input_digest=node_binary_location.digest,
            description="installing webpack"
        ),
    )
    <http://logger.info|logger.info>(process_result)

output

09:27:50.63 [INFO] Getting npm
09:27:50.63 [INFO] FrozenDict({'NVM_BIN': '/Users/nate/.nvm/versions/node/v10.15.1/bin'})
09:27:50.80 [INFO] BinaryPaths(binary_name='npm', paths=(BinaryPath(path='/Users/nate/.nvm/versions/node/v10.15.1/bin/npm', fingerprint='e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'),))
09:27:50.80 [INFO] /Users/nate/.nvm/versions/node/v10.15.1/bin/npm
09:27:50.80 [INFO] /Users/nate/.nvm/versions/node/v10.15.1/bin/node
09:27:50.81 [INFO] Completed: installing webpack
09:27:50.82 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/nate/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.zsCKZK/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/Users/nate/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.zsCKZK/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/nate/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.zsCKZK/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/Users/nate/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.zsCKZK/install/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/Users/nate/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.zsCKZK/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/nate/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.zsCKZK/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 498, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'installing webpack' failed with exit code 127.
stdout:

stderr:
env: node: No such file or directory

I'm not sure about this bit:

env={"PATH":tuple(search_path)}

, don't you want

env={"PATH": os.pathsep.join(search_path)}

?

If I decide to go the external tool route is there a way to determine what processor we are running on?

Yeah, the

Platform

type is CPU arch x OS. See

platform.py

oops! even with everything on the path it was still having trouble following symbolic links, once and if I supply the explicit path to the javascript file I need node to execute it works (instead of the symlinks in

./install_dir/bin

or

/node_modules/bin

)

glad it's working! @witty-crayon-22786 I can't recall, do we expect those symbolic links to work?

i don’t think that i understand which symbolic links these are

I will try this again with a correctly set path and i’ll have more info