https://pantsbuild.org/ logo
Powered by
s

steep-match-53634

01/16/2023, 11:15 PM
1. I want to build a dockerized app that depends on 
mysqlclient
that requires building a C library, but as Pants kinda pushes you to use pex, I'm ending up with building it not on the system that will actually run the app, and of course I'm getting a 'missing .so' error at run time. How do you solve this problem in a Pants setup? Maybe there's a way to add a dependency on top of existing pex? Or how do I prebulid a proper variant of 
mysqlclient
and make sure Pants includes it specifically (or platform dependent)? I tried building a 
pex_binary
with 
complete_platforms
that include my system's and container system's output of 
pex3 interpreter inspect --markers --tags
, but it fails with 'No matching distribution' for both.
h

happy-kitchen-89482

01/16/2023, 11:18 PM
You could pre-build wheels for 
mysqlclient
and host them on an internal pypi server
But I'm not sure about the link from "Pants pushes you to use pex" to this problem? If you weren't using pex how would this be different?
s

steep-match-53634

01/16/2023, 11:49 PM
And how does Pants then determines what wheel it should take?
I would build the library when building traditional venv (from requirements or poetry) at docker build stage and not before (when using pex)
h

happy-kitchen-89482

01/17/2023, 2:42 AM
You specify a platform or a complete platform on the pex_binary target
So you could also build the pex inside a docker container. Either by running Pants in a container, or, in 2.15, you can use the new environments feature to have Pants execute just the relevant processes in docker for you: https://www.pantsbuild.org/v2.15/docs/environments
That is probably the way to go in the near future
s

steep-match-53634

01/17/2023, 8:05 AM
Thanks, I'll look into that! It potentially solves another issue, that neither 
platform
nor 
complete_platforms
isn't available for a 
python_source
targets
h

happy-kitchen-89482

01/17/2023, 12:37 PM
That is correct - sources are sources, binaries are binaries 🙂
a binary has a target platform, a source is not platform-specific
s

steep-match-53634

01/17/2023, 1:48 PM
It sounds pretty reasonable, but how do I define a reusable python library that I can just import from with multiple build variants? I don't think 
pex_binary
will work, do I have to turn the whole thing into a python package? Using 
python_distribution
? I thought it's as easy as giving a name to your 
python_sources
and then just depending on it
Also for some reason 
docker_environment
doesn't work for me. It seems to just ignore it. I even removed the 
local_environment
definition so it doesn't have to choose. It's not even trying to pull the image. I tried both in our code and in a pretty much isolated example, with a single source that imports 
MySQLDb
(provided by 
mysqlclient
), 
pex_binary
out of it, and then a 
docker_image
that depends on the binary and sets the entrypoint + copies the pex. It fails to import inside docker and succeeds if I just run the pex without the container. The workstation is Ubuntu 22.04 and the image is based on python:3.10-buster
w

witty-crayon-22786

01/17/2023, 5:41 PM
@steep-match-53634: after declaring the environment target, did you mark the 
pex_binary
as using it? and then 
./pants package
was the command you were running?
s

steep-match-53634

01/17/2023, 6:37 PM
@witty-crayon-22786 Yes
w

witty-crayon-22786

01/17/2023, 6:38 PM
hm… would you mind filing an issue describing the setup and the failing command? that should work.
s

steep-match-53634

01/17/2023, 7:45 PM
I'll do it, thanks for reply!
4 Views
Powered by