blue-city-97042
09/07/2023, 9:07 PMpip install --use-pep517 <some wheel>
. These builds are executed in an x86 docker environment. The wheels after being built are then collected into a directory which we hope to use later on in the build process for our actual python code. Since these wheels are built in a docker environment dynamically, I was wondering if there was a way to add them as resources to a python_distribution
target. We can't currently publish them to a pypi repo, so having them available via a find_links or other mechanism is the only option we have.flat-zoo-31952
09/08/2023, 12:10 AMflat-zoo-31952
09/08/2023, 12:12 AMblue-city-97042
09/08/2023, 3:17 PMpip install --use-pep517
and some other flags to make sure they get built properly for our OS/Python distro
3. Once pants is done doing the build we'd like to do a copy of the built binaries and copy them to a location where they can be consumed by the python_distribution.
4. After this is done we'd like to have the .whl files collected and put into a location where they can be used as a python repo (using find_links
for instance)
5. Using pex distros isn't available as we need these installed as wheel files for execution on a traditional virtualenv deployment. In fact the entire app gets bundled into an rpm and deployed that way.
To confound matters further, the system we are targeting is the CentOS7/Python 38/x86 runtime, so doing builds from our MacOS machines isn't an option. Because of this I have set up a docker_environment
to do the builds. This works with the custom scripts we need to setup and build the wheel files. But the output doesn't appear to be available anywhere in the pants root or sub-folders. Additionally if we wanted to do the distro using setup.py we need to make sure that it's compiled specifically for the x86 CentOS/Python38 combo. Specifically there doesn't appear to be a way to set an environment
property on the python_distribution
target.blue-city-97042
09/08/2023, 3:58 PMflat-zoo-31952
09/08/2023, 4:20 PMpython_distribution
stuff in Pants myself as well as environments, not sure how much I can help in that context. Some thoughts:
1. The output from you docker_environment
builds should be available somehow. What target are you using that references that environment
to perform the build if python_distribution
doesn't support it?
2. For consuming the wheels with Pants from a directory you can look at your options for local requirements
3. You can use python_distribution
for building your first party code, and declare a requirement on the other wheels you've built.
There's something fundamentally non-Pantsy about having that find_links
directory there as a stateful cache that Pants isn't really managing. So yeah, I think this could be improved. It looks like you need a way to perform delcare your sdist -> wheel transformations as environment-aware Pants targets of some kind, and then be able to consume those as both requirements for your first party code, and ship them as distributions with package
goals. Is that correct?flat-zoo-31952
09/08/2023, 4:21 PMblue-city-97042
09/08/2023, 4:24 PMblue-city-97042
09/08/2023, 4:24 PMflat-zoo-31952
09/08/2023, 4:45 PMblue-city-97042
09/08/2023, 4:58 PM