Hi all, I have a question on dependency management...
# general
p
Hi all, I have a question on dependency management. In my repo there is a common requirements.txt at the root folder, and separate requirements.txt files under subfolders of each project. The common requirements.txt contains dependencies that are discoverable to all projects, whereas requirements.txt at the subfolders contain dependencies that may have conflicting versions. I have read (https://www.pantsbuild.org/docs/python-third-party-dependencies) on how to manage multiple requirements files, but it seems the only way to have conflicting versions of dependencies is to write "python_requirement_library()" for each of them. This makes the build script complex. Is there any better way to do this? What is the best practice in this case? Thanks!!
h
Hello!
but it seems the only way to have conflicting versions of dependencies is to write "python_requirement_library()" for each of them.
That wouldn't quite help. FYI, that
python_requirements()
macro is doing nothing more than reading your requirements.txt and creating
python_requirement_library()
for each entry Instead, what that warning is saying is that dependency inference won't work for those conflicting versions. You will need to explicitly add the requirement you want to the
dependencies
field wherever that requirement is used, e.g. add
dependencies=["3rdparty/python:Django"]
to the relevant
python_library
or
python_tests
target
p
You are right. It seems the only way is to write "python_requirement_library( name="old_requests", requirements=["requests==2.8.0"], )" for each of the conflicting requirements
h
Not quite - you should be able to have
requests>=3
in
project1/requirements.txt
and then
requests==2.8.0
in
project2/requirements.txt
. You could alternatively define those in BUILD files with
python_requirement_library
, but you can use
requirements.txt
instead Then, whenever you have
import requests
, you need to decide if you should add
project1:requests
or
project2:requests
to the
dependencies
field Does that make sense?
p
Ah - that is what I look for. Thanks!
❤️ 1
I am looking forward to multiple lockfiles feature in pants 2.6. "./pants package" become very slow when I have multiple requirements.txt in the repo.
💯 1