https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-father-404

03/22/2021, 8:26 PM
4210c6c9dece3ff6d0c00b318b880bd6050c3e96
resulted in
./pants lint
hanging with Black, Flake8, and Docformatter - it doesn't finish execution. Could someone please try running
./pants lint src/python/pants/util/strutil.py --no-process-execution-use-local-cache --no-pantsd
?
e

enough-analyst-54434

03/22/2021, 8:35 PM
The idea being ci lint is hitting remote cache somehow masking this bug?
h

hundreds-father-404

03/22/2021, 8:39 PM
I don't think remote caching is in play, I'm not using that locally. I only have
--no-process-execution-use-local-cache
to force the rerun
w

witty-crayon-22786

03/22/2021, 9:26 PM
trying it.
i don’t repro on main
👀 1
h

hundreds-father-404

03/22/2021, 9:40 PM
Want me to try deleting named_caches to see if it fixes it? I'd hope that I could reproduce the failure again by going back to the prior commit to warm the cache
w

witty-crayon-22786

03/22/2021, 9:51 PM
would move it aside rather than nuking, but yea, perhaps
@hundreds-father-404: but, first: can you check which processes are hung?
h

hundreds-father-404

03/22/2021, 9:51 PM
What do you mean? It's running Black, Flake8, and Docformatter. Isort completes. What am I checking?
w

witty-crayon-22786

03/22/2021, 9:52 PM
have those processes actually been forked?
if so, where are they hung? can check with
py-spy
ps -ef | grep black
, for example
👍 1
and if you want to pair on this, let me know.
h

hundreds-father-404

03/22/2021, 9:55 PM
Excellent! It's an infinite loop:
Copy code
$ /opt/homebrew/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -sE /private/var/folders/hm/qjjq4w3n0fsb07kp5bxbn8rw0000gn/T/process-execution6Y2Rzi/.cache/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/pex --check --config pyproject.toml src/python/pants/util/strutil.py

Re-execing from /opt/homebrew/opt/python@3.9/bin/python3.9
Re-execing from /Users/eric/.cache/pants/named_caches/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/bin/python3.9
Re-execing from /Users/eric/.cache/pants/named_caches/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/bin/python3.9
Re-execing from /Users/eric/.cache/pants/named_caches/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/bin/python3.9
Re-execing from /Users/eric/.cache/pants/named_caches/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/bin/python3.9
This is the process being run
w

witty-crayon-22786

03/22/2021, 9:56 PM
based on … ps?
h

hundreds-father-404

03/22/2021, 9:57 PM
Yeah,
ps -ef | grep pants
showed me the 3 processes still running, and one of them was that above one that I posted
w

witty-crayon-22786

03/22/2021, 9:57 PM
to be clear, just running something from
ps
may not actually repro precisely, because it won’t have the relevant environment variables, etc
✔️ 1
so it might be a loop, but should confirm
@hundreds-father-404: to confirm, would run with
--no-*-cleanup
👍 1
then run the
__run.sh
script, which does set env vars
👍 1
h

hundreds-father-404

03/22/2021, 9:59 PM
Great, same
re-execing
loop. Glad it's something obvious like that! So now to figure out why it's re-execing
It's something to do this with this part:
Copy code
#!/Users/eric/.cache/pants/named_caches/pex_root/venvs/short/c3eb1b0a/bin/python3.9 -sE

if __name__ == "__main__":
    import os
    import sys

    venv_dir = os.path.abspath(os.path.dirname(__file__))
    venv_bin_dir = os.path.join(venv_dir, "bin")
    shebang_python = '/Users/eric/.cache/pants/named_caches/pex_root/venvs/short/c3eb1b0a/bin/python3.9'
    python = os.path.join(venv_bin_dir, os.path.basename(shebang_python))
    if sys.executable not in (python, shebang_python):
        sys.stderr.write("Re-execing from {}\n".format(sys.executable))
        os.execv(python, [python, "-sE"] + sys.argv)
I added logging and got:
Copy code
python: /private/var/folders/hm/qjjq4w3n0fsb07kp5bxbn8rw0000gn/T/process-execution18NBX4/.cache/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/bin/python3.9
python real_path: /opt/homebrew/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin/python3.9
sys.executable: /Users/eric/.cache/pants/named_caches/pex_root/venvs/3c89ae006bab8f37897fd91a0a2d26ee196c0977/837c093c06bd686fc4cd8a7c11bdb7b9cb3598d4/bin/python3.9
sys.executable real_path: /opt/homebrew/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin/python3.9
e

enough-analyst-54434

03/25/2021, 5:23 AM
Thanks - that's useful enough data to solve if not root cause: https://github.com/pantsbuild/pex/issues/1285
❤️ 1
🎉 1