#development
- e
enough-analyst-54434
06/01/2018, 3:52 PMSince Ng runs java in a thread on server side - e
enough-analyst-54434
06/01/2018, 3:53 PMOur py impl does set environment in processes it spawns (doesn't use threads) - e
enough-analyst-54434
06/01/2018, 3:53 PMSo, our pailgun implements more functionality than nailgun - e
enough-analyst-54434
06/01/2018, 3:54 PMTo read the passed environment as a nailgun "process" in java, you need to implement a special interface where client environment gets passed in as a Map - e
enough-analyst-54434
06/01/2018, 3:55 PMYou shouldn't be happy, but hopefully that made sense - a
average-vr-56795
06/01/2018, 3:56 PMSo are you saying that we’re currently buggy because when we send commands to nailgun’d java we don’t pass the env Map? - e
enough-analyst-54434
06/01/2018, 3:56 PMYes - e
enough-analyst-54434
06/01/2018, 3:56 PMWe can't - a
average-vr-56795
06/01/2018, 3:56 PMWhy not? - e
enough-analyst-54434
06/01/2018, 3:57 PMAt base level, nailgun saves time via 1 jvm, so it dispatches via threads - e
enough-analyst-54434
06/01/2018, 3:58 PMSo it would have to mutate environment visible to all threads - e
enough-analyst-54434
06/01/2018, 3:58 PMI think that's why Ng does not do this - e
enough-analyst-54434
06/01/2018, 3:58 PMSo, instead, custom interface - e
enough-analyst-54434
06/01/2018, 3:58 PMWe could implement that interface for each binary we hit - a
average-vr-56795
06/01/2018, 3:58 PMSo we’d need to find every place that e.g. zinc reads the environment and make it use a different mechanism of reading the env? - e
enough-analyst-54434
06/01/2018, 3:58 PMBut that is totally nontrivial - e
enough-analyst-54434
06/01/2018, 3:58 PMYes - e
enough-analyst-54434
06/01/2018, 3:59 PMExactly - a
average-vr-56795
06/01/2018, 3:59 PMok 🙂 - a
average-vr-56795
06/01/2018, 3:59 PMSo the fix for now should be: Make sure we spawn the nailgun with the right env, make sure we note down the env, and tear-down and re-spawn the nailgun if the env changes, or something like that? - e
enough-analyst-54434
06/01/2018, 3:59 PMBut then nailgun only adds overhead - e
enough-analyst-54434
06/01/2018, 4:00 PMSo should be skipped - a
average-vr-56795
06/01/2018, 4:00 PM(I’m still slightly lost as to why whether our python process calls
orexecv
would make a difference to the nailgun process, here…)execve - e
enough-analyst-54434
06/01/2018, 4:00 PMRight, unless environment hash is stable - a
average-vr-56795
06/01/2018, 4:00 PMBut I need to run for a bit, so I’ll be back in a while - e
enough-analyst-54434
06/01/2018, 4:09 PMAgreed. I'm not sure what's going on here yet either. - e
enough-analyst-54434
06/01/2018, 4:14 PMThe environment variable in particular is the issue, not environment variables in general: https://github.com/pantsbuild/pants/blob/16246c8a9d84c875208a5e370da87888e799dd96/src/python/pants/java/executor.py#L194 - r
rough-minister-58256
06/01/2018, 8:31 PMso in the daemon context I’m pretty sure we never explicitly set
from the nailgun environment, which seems to effectively be the problem hereos.environ - r
rough-minister-58256
06/01/2018, 8:31 PMwe only pass that forward and reference it as a replacement foros.environ - w
witty-crayon-22786
06/01/2018, 8:31 PMmm.