I'm running into an odd issue that I haven't run i...
# general
b
I'm running into an odd issue that I haven't run into with older versions of pants that I'l hoping someone has experience with. I have a wheel that I've build to run under python 3.6, currently tagged with
cp36-none-any
which works under pants 1.30. After bumping to pants 2.11, all of a sudden pants says that the tag is no longer a listed as a viable candidate. I tried renaming the wheel to include one of the supported tags for python 3.6, however that hits the same error. Is there a good way to tell pants to either not worry about the tag, OR tag the wheel as something that's supported?
e
Two things here: 1. In general you should never be manually tagging a wheel. That should be done by the build system (
python setup.py bdist_wheel
or newer school pyproject.toml builds). 2. Leaving 1 aside, the critical thing is the Python Pants thinks your project wants to run with. If you've configured Pants with interpreter constraints that say the repo is "CPython==3.6.*" then a
cp36
wheel should work. I think the issue here is in 2. Your old Pants setup is picking a CPython 3.6 interpreter to handle your project code and the Pants v2 setup is not. Have you configured anything in Pants v2
pants.toml
to steer which Python(s) your project is expected to work with?
The clearing house for Pants v2 interpreter setup for your project is here: https://www.pantsbuild.org/docs/python-interpreter-compatibility
b
One thing that's interesting here is that the build I have going does not use bdist_wheel, but wheel itself from terminal, so maybe if I pivot then I can tag with the appropriate tags. I have set interpreter compatibility in the pants toml as I'm still constrained to 3.6 for the time being
e
"but wheel itself from terminal"
I can't parse that. Can you explain more?
b
Sorry, the method is currently
wheel convert x.y.z.egg
rather than
python setup.py bdist_wheel
e
Ah, ok. My sympathies.
If you could provide full, detailed Pants output instead of paraphrasing, that would probably be a good start.
b
Here's the full output. I have the orekit wheel from a previous step in our build process after it was converted from an egg
e
Ok, great. Thanks. So that suggests
py36-none-any
from way down the list. In other words,
cp36
doesn't make sense without further restrictions since it implies a platform specific wheel with C extensions.
none-any
contradicts that. So, stepping back, what does
orekit
consist of? Is it pure Python code? If so, can you spot if it's Python 3 code? Or does it support Python 2?
b
orekit is a java program wrapped up to be imported into python.
e
Ok, well I have no clue how that works. JNI <-> Python C api? So, first a basic fact - the wheel tag must match one of the listed wheel tags in the debug output you provided. That list of tags is the complete list supported by the python interpreter selected by Pants. You could actually pick any random tag and that case will work. The issue with picking a random tag, is when you go to run against another slightly different Python 3.6 interpreter, its tag set may be different and not include the random tag you pick; so what you want to do is pick the least specific tag (lower in the list is less specific) that you know is accurate. So, if you know the wheel does use the Python C api and its compatible with the manylinux1 standard, you'd pick
cp36-cp36m-manylinux1_x86_64
, as an example. The most specific tag in the list (1st entry) is the safest bet. That says
cp36-cp36m-manylinux_2_28_x86_64
glibc 2.28 is linked against. If you know all your target machines have glibc 2.28 / Pythons built against that glibc, you'll be good. This is all very hard to guess at though. Perhaps you can run
auditwheel
against your converted wheel and see what it says?
b
This is what it gave me. I can change the name of the wheel if needed or run the auditwheel repair job too to see what happens
e
You definitely want to let it repair.
b
Ok, I'll run the repair inside of pants with v 2.11 and see what I get. Thanks!
e
You're welcome. This is a case we've never encountered: bespoke
egg -> wheel -> auditwheel repair
; so I'd love it if you can report back success or no.
b
Yeah, I'll report back and let you know what happens
e
FWIW, Pants 1.30.0 used Pex 2.1.12 which simply wasn't doing things right - it did not check wheel tags at all; so that's why this used to work. Basically, by luck.
b
Oh, having that context certainly makes sense as to why I just started seeing it.
I finally got around to trying the repair and audit wheel shows that the proper tag is applied to the wheel after I repair the existing wheel, however pants still sees it as
cp36-none-any
which doesn't match any of the tags for my interpreter, even if I use platform
linux_x86_64
in the auditwheel repair
e
That makes little sense. Do you mind filing an issue with all the pertinent details?: + wheel name (better, upload the wheel) + pants.toml + Relevant requirements files / pyproject.toml files and BUILD files (So we can see how you reference the wheel file) + Full pants command line and output with
-ldebug
and
--pex-vebosity=9
b
I'm working on creating a ticket with all of that information, but github says the wheel is 2MB larger than the accepted limit. What's the best way to share the wheel?
I made it into a tar.gz as well, so there's hopefully some compression
e
The only interesting bits are all in the
*.dist-info/**
top-level metadata directory. If you want to trim to just that portion of the wheel, that's fine. Please include the full wheel name too though!
b
I believe I can get the egg that ends up generating the wheel if that's helpful, the egg is only 21 mb
What we do is execute the make and cmake steps for the egg since that's build from java code, then convert to wheel so pants can import it as a dependency for one of our libraries
e
That won't likely be helpful. Its the wheel name and wheel metadata that are consulted in a resolve.
I won't be around to service the ticket for ~1 month, so you might try to pull in others to look at it once you have it filed.
b
Ok, I submitted the ticket, I'll drop this in general and link our thread for context. Thanks for the help!