Hi everyone - I'm a newcomer to pants and am excit...
# general
m
Hi everyone - I'm a newcomer to pants and am excited to use it to try to tackle development of a python mono repo my company is working on! The tooling ecosystem looks first class and I can really appreciate how pants tries to bring sanity to development/build environments. A little snag on my adoption journey however is the problem of building a pex from python requirements which only have source distributions available on pypi.org. Looking at the docs for version 2.12, the
--python-no-binary
option appears to be what I want, however when trying to create a pex using the
pex_binary
target with a top-level python requirement
o365==2.0.19
(which has a binary wheel, but depends on
stringcase>=1.2.0
which only provides a tarball) I encounter a `pex.result.ResultError`when running
./pants --python-no-binary="['stringcase']" generate-lockfiles && ./pants package ::
. I am using
pants_version = "2.13.0rc3"
but when running with
-ldebug
I cannot see any evidence of
--no-binary stringcase
being set anywhere (inferred from Pull 14985). Has the
--python-no-binary
option been documented, but not implemented yet? How were people in the community solving the issue of sdists in their pex dependencies prior to this? FTR the
ResultError
traceback looks like the following:
Copy code
pex.result.ResultError: Failed to resolve compatible artifacts from lock 3rdparty/python/default.lock for 1 target:
1. cp37-cp37m-linux_x86_64:
    Failed to resolve all requirements for abbreviated platform cp37-cp37m-linux_x86_64 from 3rdparty/python/default.lock:

Configured with:
    build: False
    use_wheel: True

Dependency on stringcase not satisfied, 1 incompatible candidate found:
1.) stringcase 1.2 (via: o365==2.0.19 -> stringcase>=1.2.0) does not have any compatible artifacts:
    <https://files.pythonhosted.org/packages/f3/1f/1241aa3d66e8dc1612427b17885f5fcd9c9ee3079fc0d28e9a3aeeb36fa3/stringcase-1.2.0.tar.gz>
Note I have also tried setting the following in
pants.toml
to no joy:
Copy code
[python]
no_binary = ["stringcase"]
lockfile_generator = "pex"
...

[subprocess-environment]
env_vars.add = ["PIP_NO_BINARY=stringcase"]
Any help would be hugely appreciated 🙏
1
h
Hi, can you clarify why you need to set
no_binary
at all? If
stringcase
has no wheels then the sdist should be consumed automatically
It looks like you're using a lockfile? Have you regenerated it?
Also, the command you pasted,
./pants --python-no-binary="['stringcase']" && ./pants package ::
, looks odd - it's running pants twice, the first one is a no-op, and the second one doesn't set that flag
m
Hi @happy-kitchen-89482 - thanks for getting back 🙂 I misspelt the command, it should be `./pants --python-no-binary="['stringcase']" generate-lockfiles && ./pants package ::`(so yep using lockfiles) - I'll edit the original message to be correct
Erm to be honest I was hoping the default behaviour for building a pex would be to prefer wheels, but accept source distributions such as
*.tar*
or
*.zip
if a wheel is not available on PyPI (which is the default behaviour of pip I believe)
I've tried deleting the lockfile (mine gets generated under
3rdparty/default.lock
) and doing a fresh
./pants generate-lockfiles
but the error still occurs when doing the
./pants package
step
h
That should be the default behavior - prefer wheels but accept sdists
Oh, try setting that flag on the
./pants package
command as well?
m
That was what I was hoping you would say for package resolution 😉 Yep I did try that too (i.e.
./pants --python-no-binary="['stringcase']" package ::
but it didn't seem to have any affect either way... Would it help if I provided a snipped for the
pex_binary
within my
BUILD
?
h
can't hurt!
m
Gotcha, the (simplified with relevant parts)
BUILD
file for the pex is:
Copy code
# type: ignore

python_sources(
    name="statics",
    dependencies=[
        "src/python/apps/myapp/templates",
        "src/python/apps/myapp/assets",
    ],
)

dependencies = [
    "3rdparty/python:requirements",  # python_requirements target pointing to requirements.txt with o365
    ":statics",
    "src/python/libs/mylib:mylib_sources",
    "src/python/apps/myapp/myapp:myapp_sources",
]

pex_binary(
    name="gunicorn_run",
    description="Self-contained Django executable served with gunicorn",
    entry_point="gunicorn_run.py",
    include_requirements=True,
    platforms=["linux-x86_64-cp-3.7.13-m"],  # NOTE: previously 3.7.5
    dependencies=dependencies,
    emit_warnings=True,
    restartable=True,
)

file(name="server", source="start-server.sh")
h
Not sure if this is relevant, but you shouldn't need that explicit
"3rdparty/python:requirements"
dependency
Pants will infer deps on the subset of your requirements that are actually needed
And if you need to add an explicit dep that couldn't be inferred for some reason, then it's e.g.,
3rdparty/python:o365
Pants generates targets for each requirement in requirements.txt
But apart from that this seems like it should work
h
*`3rdparty/python#o365` on Pants 2.12
h
Oh, wait a minute
Re
platforms=["linux-x86_64-cp-3.7.13-m"]
Which platform are you running on?
Since there is only an sdist for
stringcase
, if you're not running on a
linux-x86_64-cp-3.7.13-m
platform then you can't build a compatible wheel on your platform
m
I'm running on WSL Ubuntu 20.04 on an modern Intel machine if that affects behaviour and have my interpreter set as follows in `pants.toml`:
Copy code
[python]
interpreter_constraints = ["==3.7.13"]
Removing both the explicit
3rdparty/python:requirements
dependency and
platforms
causes pants to work 🙌 I cannot say how grateful I am as I've been really hitting a brick wall on this! Thanks for all of your help 😀
❤️ 3
🔥 1