https://pantsbuild.org/ logo
Join Slack
Powered by
Hi, any ideas on how to debug a segfault in CI wit...
# development
h
Hi, any ideas on how to debug a segfault in CI with building Pants wheels using Py39? It was happening with Travis and still happens with GitHub Actions. Only on Linux. I can't reproduce locally, even when using the centos7 and manylinux2014 docker images used by the respective CI envs https://github.com/pantsbuild/pants/pull/11858/checks?check_run_id=2329416891#step:9:1972
Oh interesting, in macOS it gives 
7114 Illegal instruction: 4
https://github.com/pantsbuild/pants/pull/11858/checks?check_run_id=2329416813#step:9:1733
e
A quick scan of packages.py only nets xml as a likely native culprit. Might try modifying packages.py to dump xml to console before parsing / switch xml with lxml, etc.
🙌 1
Maybe defusedxml - that's pure python and modernish fwict.
h
Ah, will try upgrading our 3rdparty deps too. Probably good to do either way
Interesting. Things work if you only build Py39 without Py37 and Py38, implying there is some type of corrupted leftover state...
w
ew.
h
I wonder if this might explain some of the flakes in CI we've been getting with illegal instructions?? Hopefully this PR will uncover something we fix, and would be awesome if this fixes that too
w
the python interpreter is supposed to be set/overridden in the rust build by an env var
👍 1
i’m trying to locate it, but failing.
h
we do 
python --version
, I think after sourcing the venv. 
calculate_engine_hash.sh
iirc
w
yea, but it’s not clear what cargo is consuming
we used to set something in 
cargo.sh
…ah.
Copy code
PYTHON_SYS_EXECUTABLE
👀 1
👍 1
h
NB that in the CI logs where I did Py37-Py39 together, the compilation was only 15 seconds and only of engine crate, whereas I would have expected more to have compiled
w
only of engine crate
hm… yes. i was going to say “that makes sense”, but i think that 
python3-sys
would need to recompile as well.
👍 1
i think that i see how this could happen. one sec.
🙌 1
h
Huh, and it works with Py38 and Py39, but not Py37, Py38, and Py39 👀 Will test Py37 and Py39
w
maybe not surprising if they didn’t change the ABI that we use…?
❤️ 1
you can use that branch in your build like this:
Copy code
diff --git a/src/rust/engine/Cargo.toml b/src/rust/engine/Cargo.toml
index 6d897403f..e7797d2a3 100644
--- a/src/rust/engine/Cargo.toml
+++ b/src/rust/engine/Cargo.toml
@@ -102,7 +102,7 @@ async_semaphore = { path = "async_semaphore" }
 async-trait = "0.1"
 bytes = "1.0"
 concrete_time = { path = "concrete_time" }
-cpython = "0.5"
+cpython = { git = "<https://github.com/stuhood/rust-cpython>", rev = "46d7eff26a705384e41eb6f7b870cd3f5f14b3bc" }
 crossbeam-channel = "0.4"
 fnv = "1.0.5"
 fs = { path = "fs" }
🙌 1
h
That fixed it!! Thank you @witty-crayon-22786 ❤️ Py39 is now ready to land
w
let me push that branch to a pants fork before you do
👍 1
ok, pushed to
Copy code
cpython = { git = "<https://github.com/pantsbuild/rust-cpython>", rev = "46d7eff26a705384e41eb6f7b870cd3f5f14b3bc" }
👍 1
and in a comment above, should reference the PR: https://github.com/dgrunwald/rust-cpython/pull/261
👍 1
Open in Slack
PreviousNext
Powered by