pantsbuild #general

if i depend on 3rdPartyA and 3rdPartyB, and 3rdPartyB -> 3rdPartyA (with a version compatibility match), then npm install will only install 3rdPartyA and 3rdPartyB, and not put another copy of 3rdPartyA under 3rdPartyB, as it would if i depended on 3rdPartyB only

witty-crayon-22786

08/27/2015, 12:17 AM

if A and B are in 3rdparty, they would be installed separately.

witty-crayon-22786

08/27/2015, 12:17 AM

so you would get precisely those versions

witty-crayon-22786

08/27/2015, 12:17 AM

and they would be cached

witty-crayon-22786

08/27/2015, 12:17 AM

right?

bright-dinner-81701

08/27/2015, 12:17 AM

i.e. package.json of { “3rdPartyA”, “3rdPartyB” } yields project node_modules 3rdPartyA 3rdPartyB while package.json of { “3rdPartyB” } yields project node_modules 3rdPartyB node_modules 3rdPartyA

bright-dinner-81701

08/27/2015, 12:18 AM

ideally i think yes

bright-dinner-81701

08/27/2015, 12:18 AM

but

bored-art-40741

08/27/2015, 12:18 AM

This is a situation where you really have to differentiate resolution and installation

bright-dinner-81701

08/27/2015, 12:18 AM

i dont know how to get NPM to used a cached copy like that

bored-art-40741

08/27/2015, 12:18 AM

We have similar maladies in python

witty-crayon-22786

08/27/2015, 12:18 AM

the package.json would be full of symlinks, not actual remote deps

rough-minister-58256

08/27/2015, 12:18 AM

need a new .nex format

bright-dinner-81701

08/27/2015, 12:35 AM

@witty-crayon-22786: the thing is, to install A and B “separately” you need to be in a totally different folder for each one (or specify that theyre being installed globally with -g and specify a different —prefix for each one, as @enough-analyst-54434 suggested, to basically trick NPM into giving you an isolated environment for that particular installation), because otherwise they just get installed into node_modules under whatever folder youre in, with their dependency installations coupled as described above (the https://docs.npmjs.com/cli/install#algorithm algorithm)

enough-analyst-54434

08/27/2015, 12:35 AM

My branch-as-RB may be a helpful reality to base discussion around

enough-analyst-54434

08/27/2015, 12:36 AM

There is some level of handwave going on for us non-npm-minded folks at the moment

enough-analyst-54434

08/27/2015, 12:36 AM

ie: my unit tests will be pointable-at to say - there that's not a good thing to do

rough-minister-58256

08/27/2015, 12:37 AM

you could use an approach similar to how thriftgen works today - install each one in an isolated dir and then merge them.

bright-dinner-81701

08/27/2015, 12:37 AM

also, package.json files are just instructions on where to find things for installation, not runtime resolution, and i think even if you do the above and then make a bunch all your source modules’ package.json files point to those cached copies using symlinks or relative paths, it will still “install” them again by copying them into place under your package, defeating the point of installing them earlier and caching

enough-analyst-54434

08/27/2015, 12:37 AM

Yeah - alluded to this with Chris offline

enough-analyst-54434

08/27/2015, 12:38 AM

I think correctness or npm emulation - ie: don't surprise the user, is 1st order of business

bright-dinner-81701

08/27/2015, 12:38 AM

yeah

enough-analyst-54434

08/27/2015, 12:38 AM

We can worry about cahing speed stuff more in rev 2...N

bright-dinner-81701

08/27/2015, 12:38 AM

yeah which is why i ended up in the doc saying that you just build a big copy of the sources under a working dir in .pants.d and do an npm install at the end

enough-analyst-54434

08/27/2015, 12:39 AM

I've already observed that twiddling both the cache dir for npm and the min cache time for its unbounded resolves goes a long way towards reasonably fast re-installs in the steady state

bright-dinner-81701

08/27/2015, 12:39 AM

essentially, i just dont know how to make npm properly use prebuilt/preinstalled node modules for certain things