Bummer new venv mode broke running Black over gt 1 file for

Question

Bummer new venv mode broke running Black over gt 1 file for me Going back to `652df2c` works when running ` pants lint src python pants util py` but master branch fails with

hundreds-father-404 · Answer

Untitled

hundreds-father-404 · Answer

cc < enough analyst 54434> yay macOS At least this reproduces with strictly Pex no Pants

enough-analyst-54434 · Answer

Why do you think this is a Pex issue

hundreds-father-404 · Answer

Vs a Black issue It could be although it s only happening when running via ` venv` mode not ` unzip` and zip app

enough-analyst-54434 · Answer

OK Its not at all clear Pex has anything to do with this Since there are 0 concurrency issues while Pex is running the backtrace is pure venv

enough-analyst-54434 · Answer

I ll dig a bit and probably push back on the issue without more proof

enough-analyst-54434 · Answer

< witty crayon 22786> and < hundreds father 404> please try when you get a chance Alternatively ```pex c black black==20 b1 venv o black pex PEX TOOLS=1 black pex venv black venv black venv bin black pex py```

enough-analyst-54434 · Answer

The error is in multiprocessing and mp has weird rules surrounding ` main ` the bin scripts in a venv all have a ` name ` guard on ` main `

witty-crayon-22786 · Answer

mm so Eric pinged me about this and i added some details to <https github com pantsbuild pex issues 1235> in case you hadn t seen them

witty-crayon-22786 · Answer

< enough analyst 54434> and that would be pex 2 1 30

enough-analyst-54434 · Answer

I di unuseful for someone without macOS

enough-analyst-54434 · Answer

Well yeah You two claimed `pex ` foo on that ticket

enough-analyst-54434 · Answer

So whatever pex you both used

witty-crayon-22786 · Answer

`20 8b1`

witty-crayon-22786 · Answer

one sec

hundreds-father-404 · Answer

Yay your Pants Pex fixes it < witty crayon 22786> nb that you should set ` interpreter constraint= CPython==3 9 ` for the Pex command and `PANTS BLACK INTERPRETER CONSTRAINTS= CPython gt =3 9 ` for Pants

witty-crayon-22786 · Answer

~red herring~

enough-analyst-54434 · Answer

binary is the Benjy bug

enough-analyst-54434 · Answer

Please `rm rf ~ pex isolated`

witty-crayon-22786 · Answer

that worked thanks

enough-analyst-54434 · Answer

Worked as in black now works

enough-analyst-54434 · Answer

gt Yay your Pants Pex fixes it Eric the PR fixes it or the `PEX TOOLS=1 ` experiment

witty-crayon-22786 · Answer

success ```$ ~ src venvs pex 2 1 30 bin pex c black black==20 8b1 interpreter constraint= CPython==3 9 venv o black pex $ PEX TOOLS=1 black pex venv black venv $ black venv bin black ~ src pex py All done 61 files left unchanged ```

enough-analyst-54434 · Answer

OK So PEX TOOLS=1 POC works That implies PR works

enough-analyst-54434 · Answer

Which also implies its a ` name == main ` issue

hundreds-father-404 · Answer

Both fix it The PR works for me and the Pex snippet works

hundreds-father-404 · Answer

Thanks for digging into this John

enough-analyst-54434 · Answer

So I suspect the right fix is to add a guard early exit here <https github com pantsbuild pex blob a2e78f8402241e1100d9757d9b3c76adde1189db pex tools commands venv py L150>

enough-analyst-54434 · Answer

Could one of you try hacking such a thing in in your named caches venv pex script

hundreds-father-404 · Answer

Happy to although I m not following what that guard would look like Sorry I feel obtuse here

enough-analyst-54434 · Answer

```if name = main return```

enough-analyst-54434 · Answer

The opposite of the usual guard just to save a mega indent as you are wont

hundreds-father-404 · Answer

can t return because it s not a function but I did this on what corresponds to line 150 ``` 11 if name = main 12 sys exit 0 ``` Resulting in ```error cannot format private var folders hm qjjq4w3n0fsb07kp5bxbn8rw0000gn T process executionAkW4J8 src python pants pantsd pants daemon py A process in the process pool was terminated abruptly while the future was running or pending ``` This is on the master branch Lmk if it d be helpful to VC about this

enough-analyst-54434 · Answer

How about just try a vanilla `if name == main ` guard just under the shebang an indent the whole file under that

enough-analyst-54434 · Answer

Safe importing of main module is what we re working around here if you can take some time to fiddle <https docs python org 3 library multiprocessing html the spawn and forkserver start methods> If $ ing Apple would license their $ OS to be used under a vm running on Linux I could be more useful They won t take my money though

hundreds-father-404 · Answer

Okay great this worked on the master branch gt How about just try a vanilla if name == main guard just under the shebang an indent the whole file under that

enough-analyst-54434 · Answer

$ OK Thanks for your help I ll spin up a Pex fix for this

enough-analyst-54434 · Answer

< hundreds father 404> one more favor if you can ```for py in python3 8 9 do echo n $py amp amp $py c import multiprocessing print multiprocessing get start method done python3 8 fork python3 9 fork``` You should see a `spawn` for python3 9

hundreds-father-404 · Answer

Confirmed that 3 7 is fork and 3 9 is spawn I can t get 3 8 to install via pyenv because arm arch upside down face

enough-analyst-54434 · Answer

OK thank you very much