https://pantsbuild.org/ logo
#general
Title
# general
n

narrow-vegetable-37489

08/21/2021, 9:44 PM
Is it possible to manually define additional dependencies for a third-party dependency somehow? I have a third-party dependency (
mongomock
) that requires
setuptools
to work, but it isn't defined as a requirement and thus not added to the pex by Pants. Right now I work around it by explicitly adding
setuptools
as a dependency to all the
python_library()
with code that uses
mongomock
, but it would be nice if I somehow could tell Pants to always pull in
setuptools
when something wants
mongomock
.
f

fast-nail-55400

08/21/2021, 10:11 PM
I imagine you could manually define a
python_requirement_library
for
mongomock
and add a dependency on
setuptools
Unclear how this would interact with the
python_requirements
macro and any
python_requirement_library
it makes for
mongomock
(
python_requirements
creates
python_requirement_library
targets in its implementation and does not appear to have a way to exclude a requirement from being turned into a target in favor of your manual creation of the `python_requirement_library`target)
(assuming of course that your project uses
python_requirements
)
n

narrow-vegetable-37489

08/21/2021, 10:30 PM
Yeah, we do use
python_requirements
, so I imagine it clashing with
python_requirement_library
one way or another. Worth giving it a shot though and see what happens :)
f

fast-nail-55400

08/22/2021, 1:16 AM
if there ends up being a conflict, straightforward enough to do a PR to modify
python_requirements
to support excluding certain packages
h

happy-kitchen-89482

08/22/2021, 4:35 AM
You could remove
mongomock
from the
requirements.txt
that
python_requirements
references?
And then define it as s
python_requirement_library()
in the same BUILD file, so it's still available as
mongomock
in that BUILD file
Unless you need it in the
requirements.txt
for other reasons?
e

enough-analyst-54434

08/22/2021, 9:44 PM
Concretely, this leads to a passing test in the Pants repo:
Copy code
^jsirois@gill ~/dev/pantsbuild/jsirois-pants (main *) $ cat 3rdparty/python/BUILD.mongomock 
python_requirement_library(
  name="mongomock-fixed",
  requirements=["mongomock", "pymongo"],
  dependencies=["3rdparty/python:setuptools"],
)

^jsirois@gill ~/dev/pantsbuild/jsirois-pants (main *) $ cat src/python/pants/mongomock/BUILD 
python_tests(name='tests')
^jsirois@gill ~/dev/pantsbuild/jsirois-pants (main *) $ cat src/python/pants/mongomock/mongomock_test.py 
import mongomock


@mongomock.patch(servers=(('<http://server.example.com|server.example.com>', 27017),))
def test_test():
    pass
For
./pants test src/python/pants/mongomock/mongomock_test.py
.
💪 1
In the Pants case, we already have a setuptools requirement defined in `3rdparty/python/requirements.txt`; so this setup re-uses that. If you didn't already have a
setuptools
requirement defined anywhere else, you could just add it to the list of
requirements
in the
python_requirement_library
.
h

happy-kitchen-89482

08/23/2021, 12:10 AM
And note that you can provide version constraints and so on in the
requirements=[...]
bit.
n

narrow-vegetable-37489

08/23/2021, 11:16 AM
Thanks everyone. Currently we generate a
requirements.txt
from a
pyproject.toml
so Poetry's venv can still be used for the dependencies in PyCharm etc. - so we could simply remove
mongomock
from the
requirements.txt
and add it manually with
python_requirement_library
as you suggest. We do however currently use the same
requirements.txt
as the source for
requirement_constraints
, and that would still require
mongomock
in the file to work, correct? We could generate two files I suppose and remove
mongomock
from one of them, but the plan was to migrate to the new
poetry_requirements
macro, but I get the feeling that it would at this point cause more harm than good if I want to make my
mongomock
life eaiser 🙂
e

enough-analyst-54434

08/23/2021, 1:57 PM
@narrow-vegetable-37489 how do you generate
requirement_constraints
? Do you use (loose) requirements straight from requirements.txt or do you use a script like the one suggested here to generate a pinned constraints file to simulate a lock file? I'm guessing the former. The latter though would work with part of your requirements being in requirements.txt and part being in
python_requirement_library
targets.
n

narrow-vegetable-37489

08/23/2021, 2:03 PM
I use
poetry export --without-hashes
for the requirements.txt. If I export with hashes in a separate file for
requirement_constraints
and then run for example
./pants fmt ::
, Pants (or rather pip) complains that hashes are required for all dependencies and
docformatter
does not have any - but that's a separate issue 🙂
e

enough-analyst-54434

08/23/2021, 2:14 PM
Ok. No simple solution then that I can think of.
h

happy-kitchen-89482

08/23/2021, 7:27 PM
You can include a
requirements.txt
in another one with
-r
right? And I think Pants will ignore those lines? Maybe there's a workaround there somehow