hey folks was talking to about the en

Question

hey folks was talking to < hundreds father 404> about the env var proposall here and while i have no issues with it was wondering if there had been any consideration to preserve the entire original environment This technically does not violate hermeticity and can be quite useful for CI CD environments that load multiple credentials or other things via env

witty-crayon-22786 · Answer

any env var that was included would change the cache key of the running process

witty-crayon-22786 · Answer

and so your various users might not hit shared remote caches

witty-crayon-22786 · Answer

so it does affect hermeticity a bit

wonderful-iron-54019 · Answer

sure but affect = violate right definitely would trigger more cache misses

witty-crayon-22786 · Answer

but that s why that ticket suggests configuring which ones you want to include it s definitely useful you just don t want random stuff like this one in there ```$ env | grep LaunchInstance LaunchInstanceID=BFA89F91 2DC0 4950 BA60 5670917DE455```

witty-crayon-22786 · Answer

possibly literally random i don t know why that is in my env sweat smile

wonderful-iron-54019 · Answer

sure that makes sense OTOH we load CI CD utils in jenkins from another repo controlled by another team and then one from our team so to add something to our CI CD environment would require a change in possibly three places

wonderful-iron-54019 · Answer

seems very likely someone will forget to tack on the option

wonderful-iron-54019 · Answer

definitely agree with the risks involved but wonder if its a useful option to have while defaulting to off

witty-crayon-22786 · Answer

< wonderful iron 54019> yes in a tree falling in the forest situation env vars might not affect the run from a safety first perspective we ve tried to bias toward opting in to including things rather than opting in to exclude them in both cases you might need to whack moles to include exclude stuff but one is a safer default maybe

hundreds-father-404 · Answer

I wonder if this would be slightly more tolerable if it was an env allowlist You could set ` pex extra path= FOO BAR ` and that would signal to read from `os env` rather than having to set ` FOO= $ FOO ` You could also explicitly set the env var if you want Either way I think this is a better design than the proposal

witty-crayon-22786 · Answer

but also consider that hermeticity might be even more important in CI environments where you need more trust in the results and are more likely to be running everything that hasn t changed since the last run and thus be more interested in hitting caches

witty-crayon-22786 · Answer

yea an allowlist would be a good idea possibly even overridable per test

witty-crayon-22786 · Answer

sorry i think that i thought that that was already what the ticket was suggesting should have bothered to read it

hundreds-breakfast-49010 · Answer

I like the idea of specifying env vars a user would like a test to have access to via a command line option

hundreds-breakfast-49010 · Answer

I m not sure if it should be pex specific

hundreds-father-404 · Answer

vs applying to any subprocess ever run right We don t have a solid approach right now for where options like that should live I think you re right that we need it to apply to more than Python though For example the Protobuf subprocess should use the same env vars

wonderful-iron-54019 · Answer

yeah concur with that we have a few custom plugins that only marginally deal with python we d want to have this feature

hundreds-breakfast-49010 · Answer

maybe the scope of this option should be any subprocess pants spawns

hundreds-breakfast-49010 · Answer

or is that too broad

witty-crayon-22786 · Answer

although we ll certain need some infrastructure support that would allow rules to set these the scope of the setting should definitely just be tests i think

witty-crayon-22786 · Answer

`run` and `repl` are in the foreground and don t need this setting at all should just be able to use any env vars without a setting

witty-crayon-22786 · Answer

oh but the original usecase was during pex building ie to pass to PEX pip

hundreds-father-404 · Answer

Ditto that ` pex executable search paths` might be the wrong place for where you control the $PATH env var Consider if you are using a Protobuf plugin and that for some reason needs to discover something via the $PATH I wonder if that option should apply to everywhere

witty-crayon-22786 · Answer

< hundreds father 404> yea that s covered on the followup PATH ticket

hundreds-father-404 · Answer

gt oh but the original usecase was during pex building ie to pass to PEX pip Yeah the original motivating case was so that a test could consume the env var Which might imply we want a ` pytest extra env` option

witty-crayon-22786 · Answer

the test case is clearer i think unsure whether it should be allow vs set this literal value though

hundreds-father-404 · Answer

Vs either or You can use the shorthand but can also set the literal value

witty-crayon-22786 · Answer

oh dangit another i haven t read the ticket sorry

hundreds-father-404 · Answer

I agree in general with trying to limit this to `test` for now as it s easier to loosen things than to take away functionality So long as that solves JP s issue

witty-crayon-22786 · Answer

yep i like the syntax on the ticket

witty-crayon-22786 · Answer

is that the syntax that `tox` uses

hundreds-father-404 · Answer

Indeed `tox` is what inspired the allowlist shorthand I think they have separate settings for literal vals vs allowlist iirc though