https://pantsbuild.org/ logo
Join Slack
Powered by
How could I further debug this?
# general
c
How could I further debug this?
h
@enough-analyst-54434 any ideas?
c
Actually, I figured this out by debugging the pex resolver. So, although there is pyarrow 0.8.0 macosx version supporting my architecture, but there is no version of the package which supporting Big Sur so each alternative is being filtered out.
I think at this point this setup is so old the dependencies slowly fading out as the os keeps changing.
Please ignore the question and thank you for the consideration!
h
Yeah, this is the equivalent of booting up your parents' old laptop that you found in their basement and realizing that Windows XP doesn't really work in 2021. This may or may not have been how I spent my morning...
😅 3
😭 2
c
Unfortunately, I need to make this Windows XP work...
So, I grabbed all the pants python dependencies and build the wheels on my local machine.
Which was a bit painful as I had to patch the C code of the old versions of 
psutil
and 
python-sasl
here and there to ensure that it Big Sur compatible (even the C POSIX API diverged in the meantime on mac).
So now I have a set of locally built wheels that work.
I thought I tricked 
pants
into picking up my locally build pants, but unfortunately in the case of C hacked packages it tries to rebuild it from source (which is not patched) so it fails.
Is there a way to instruct pants to prefer binary packages of wheels?
Also I build these wheels as editable wheels, could that cause any problem during package resolution in pants?
Oh, wait now it tries to pick up an even earlier version of 
psutil
which I don't have among my pre-built wheels (yet). Ah, let me build one. If this surgery ever be successful I will be surprised 🙂
Ah, no, this is still an issue. I have a following resolvables in my pex resolver: 
[WheelPackage(u'file:///Users/rormandi/src/data-processing/wheelhouse/psutil-4.4.2-cp27-cp27m-macosx_11_2_x86_64.whl'), SourcePackage(u'<https://artifactory.url.com/artifactory/api/pypi/pypi/packages/packages/6c/49/0f784a247868e167389f6ac76b8699b2f3d6f4e8e85685dfec43e58d1ed1/psutil-4.4.2.tar.gz#sha256=1c37e6428f7fe3aeea607f9249986d9bb933bb98133c7919837fd9aac4996b07'),...>]
Then, why does this prefer source package (and start building the source package which, since it is not patched, fails with the followings)
Copy code
**** Failed to install psutil-4.4.2 (caused by: NonZeroExit("received exit code 1 during execution of `[u'/Users/rormandi/.pyenv/versions/2.7.16/envs/data-processing-env/bin/python', '-', 'bdist_wheel', '--dist-dir=/var/folders/f6/msrc3w_903vb9clkzfzhnrnh0000gq/T/tmpQtpUEI']` while trying to execute `[u'/Users/rormandi/.pyenv/versions/2.7.16/envs/data-processing-env/bin/python', '-', 'bdist_wheel', '--dist-dir=/var/folders/f6/msrc3w_903vb9clkzfzhnrnh0000gq/T/tmpQtpUEI']`",)
):
stdout:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-11.2-x86_64-2.7
creating build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_pswindows.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_common.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/__init__.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_psosx.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_psbsd.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_pslinux.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_compat.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_psposix.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
copying psutil/_pssunos.py -> build/lib.macosx-11.2-x86_64-2.7/psutil
creating build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/runner.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/__init__.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_process.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_system.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_memory_leaks.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.macosx-11.2-x86_64-2.7/psutil/tests
running build_ext
building 'psutil._psutil_osx' extension
creating build/temp.macosx-11.2-x86_64-2.7
creating build/temp.macosx-11.2-x86_64-2.7/psutil
creating build/temp.macosx-11.2-x86_64-2.7/psutil/arch
creating build/temp.macosx-11.2-x86_64-2.7/psutil/arch/osx
clang -fno-strict-aliasing -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/libffi/include -I/usr/local/opt/openssl/include -I/usr/local/opt/mysql@5.7/include/ -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=442 -DPSUTIL_OSX=1 -I/Users/rormandi/.pyenv/versions/2.7.16/envs/data-processing-env/include/python2.7 -c psutil/_psutil_osx.c -o build/temp.macosx-11.2-x86_64-2.7/psutil/_psutil_osx.o

stderr:
psutil/_psutil_osx.c:304:20: error: implicit declaration of function 'getpagesize' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    int pagesize = getpagesize();
                   ^
psutil/_psutil_osx.c:719:20: error: implicit declaration of function 'getpagesize' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    int pagesize = getpagesize();
                   ^
psutil/_psutil_osx.c:759:20: error: implicit declaration of function 'getpagesize' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    int pagesize = getpagesize();
                   ^
3 errors generated.
error: command 'clang' failed with exit status 1
Any idea how I could convince 
pants
to use the pre-built wheel package as opposed to the source package?
Since I'm hacking I've ended up excluding the source package in this case in the Pex resolver's source code...
Seems to be working or at least this is the longest python build with pants so far which is a good sign.
Thank you all for the help!
e
Ok. Assuming you meant Pants 1.15.0, that corresponds to Pex 1.5.3 which is quite old and, at that point, still using a custom resolver. Pex has long since switched to using Pip internally. Thanks for digging in and hacking on code here. Both pieces of software you're using are definitely unsupported except via your own contributions. If there is anything we can do to help get you on newer versions, please speak up.
Open in Slack
PreviousNext
Powered by