Followup to point up I m having trouble getting tests passin Pants #development

Followup to :point_up: , I’m having trouble gettin...

gentle-wolf-58752

03/07/2019, 5:26 PM

Followup to ☝️ , I’m having trouble getting tests passing on master after some of the latest changes (unsure which changes may have done it or if there’s something I need to update to get past it - macOS 10.14.3) cmd

/pants test tests/python/pants_test/build_graph:target -- -k=test_has_sources

res:

Copy code

<reduced output for brevity>
E       ImportError: dlopen(/var/folders/fv/9ly0y0dj1g946plb36nm4s1m0000gn/T/tmpxvJsvr/native_engine.so, 2): Symbol not found: __Py_Dealloc
E         Referenced from: /var/folders/fv/9ly0y0dj1g946plb36nm4s1m0000gn/T/tmpxvJsvr/native_engine.so
E         Expected in: flat namespace
E        in /var/folders/fv/9ly0y0dj1g946plb36nm4s1m0000gn/T/tmpxvJsvr/native_engine.so

Seems to be coming from

make_target

TestBase

hundreds-father-404

03/07/2019, 5:27 PM

Oof. So this is an issue when trying to run a native engine that was built with Py3 but running it with Py2

aloof-angle-91616

03/07/2019, 5:27 PM

i've had that happen recently on my arch linux laptop and i solved it by blasting everything in

/tmp

rm -rf ~/.cache/pants/bin/native-engine

, then

git clean -xfd

aloof-angle-91616

03/07/2019, 5:27 PM

aaaaaand eric just solved it

aloof-angle-91616

03/07/2019, 5:28 PM

one thing to note is that we write the file

src/python/pants/engine/native_engine.so

after building, not sure if that's something we need to clean up as well when switching pythons

hundreds-father-404

03/07/2019, 5:28 PM

Eh did I? I thought we had set things up so people would never run into this issue again, so I’m concerned about this. https://github.com/pantsbuild/pants/issues/6985

aloof-angle-91616

03/07/2019, 5:28 PM

well, i may not have run clean-all in between switching lol

hundreds-father-404

03/07/2019, 5:29 PM

Oh wait, @gentle-wolf-58752 did you run

./pants clean-all

? That is one case where this would fail if you forgot to

gentle-wolf-58752

03/07/2019, 5:30 PM

I thought I had, giving it another shot

aloof-angle-91616

03/07/2019, 5:31 PM

(i thought i had too, so i wouldn't second-guess yourself too much)

aloof-angle-91616

03/07/2019, 5:32 PM

i didn't spend enough time trying to repro the issue but i can do that the next time i run into it

👍 2

gentle-wolf-58752

03/07/2019, 5:36 PM

Hit this on the

clean-all

Exception caught: (pex.resolver.Untranslateable)

- seeing a few issues related so looking more into those

😢 1

gentle-wolf-58752

03/07/2019, 5:38 PM

Same trace as https://github.com/pantsbuild/pants/issues/6638

hundreds-father-404

03/07/2019, 5:40 PM

what happens with

PEX_VERBOSE=9

gentle-wolf-58752

03/07/2019, 5:43 PM

Quite a bit, though that was having both

2.7.14

and

3.6.6

set via

pyenv

gentle-wolf-58752

03/07/2019, 5:44 PM

Using just

3.6.6

~~just returns the same trace as noted on the issue~~ kicks back

Copy code

'Exception message: received exit code 127 during execution of ['/Users/schmitt/.pyenv/shims/python2.7'] while trying to execute `['/Users/schmitt/.pyenv/shims/python2.7']

🤔 1

hundreds-father-404

03/07/2019, 5:45 PM

You should be able to have

pyenv global 2.7 3.6 3.7

enabled just fine.

gentle-wolf-58752

03/07/2019, 5:47 PM

There’s quite a bit of output running the same test under

PEX_VERBOSE=9

- not sure what would be most helpful (seems like it’s too much for a post on the issue)

gentle-wolf-58752

03/07/2019, 5:48 PM

Guessing:

Copy code

Under current interpreter PythonInterpreter('/Users/schmitt/Workspace/pants/build-support/pants_dev_deps.py36.venv/bin/python3' ...

and

Copy code

To match given interpreter PythonInterpreter('/Users/schmitt/Workspace/pants/build-support/pants_dev_deps.py36.venv/bin/python3', PythonIdentity('cp', 'cp36m', '36', 3, 6, 6), {('setuptools', '40.4.3'): '/Users/schmitt/.cache/pants/python_cache/interpreters/CPython-3.6.6/setuptools-40.4.3-py3.6.egg', ('wheel', '0.31.1'): '/Users/schmitt/.cache/pants/python_cache/interpreters/CPython-3.6.6/wheel-0.31.1-py3.6.egg'}).

hundreds-father-404

03/07/2019, 5:49 PM

Yeah so that means it’s using Py3.6.6 during that run. I’m confused, are you getting the _Py_dealloc issue here?

gentle-wolf-58752

03/07/2019, 5:50 PM

No that was resolved with the

clean-all

(sorry should have been more clear above that this was a different / new issue, which is what sent me back to github to find other reports)

hundreds-father-404

03/07/2019, 5:50 PM

Oh okay well I’m happy _Py_dealloc is resolved! I unfortunately don’t know much about the other issue 😕 but that is good news there’s no new issue with _Py_dealloc at least

👍 1

gentle-wolf-58752

03/07/2019, 5:59 PM

Fixed it setting the flags mentioned here https://github.com/pyca/cryptography/issues/3489#issuecomment-312607156 - will note that on #6638 in pants as well

hundreds-father-404

03/07/2019, 6:00 PM

Ah yes I have this permanently exported in my .zshrc. Didn’t put it together this was the issue!

gentle-wolf-58752

03/07/2019, 6:05 PM

No worries, I just added them there too - no more issues for me here 🙂 thanks for help @hundreds-father-404 @aloof-angle-91616 !

❤️ 2

witty-crayon-22786

03/07/2019, 6:28 PM

i'm beginning to think that Twitter has been on the sidelines for a bit too long on the Python 3 front... we ought to help with the endgame.

witty-crayon-22786

03/07/2019, 6:29 PM

unfortunately, the next thing we have planned is actually a push to enable

pantsd

by default, and we need to prioritize that first for metrics purposes =(

witty-crayon-22786

03/07/2019, 6:30 PM

can continue that discussion elsewhere, but: sorry for not helping as much as we should with this effort

aloof-angle-91616

03/07/2019, 10:47 PM

the pantsd push is only relevant for the people working on pantsd

witty-crayon-22786

03/07/2019, 10:48 PM

heh. true. there are other pushes happening.

Open in Slack

Previous Next