modern-monkey-78364
12/16/2022, 7:38 AMpython_sources()
python_tests(
name="tests",
)
And pants used to figure out all the dependencies. But now after the upgrade, all the test cases with above BUILD files are failing since pants is not able to figure out the dependencies. Is there a setting I am missing or this is no longer supported?happy-kitchen-89482
12/16/2022, 7:56 AMmodern-monkey-78364
12/16/2022, 8:43 AMpython_tests(
name="dummy_test.py",
sources=["test_dummy.py"],
dependencies=[":dum"],
)
But now it throws exception:
ModuleNotFoundError: No module named 'pkg_resources'
Even though I have this in my project BUILD file
python_requirement(
name="setuptools",
requirements=["setuptools"],
)
My pants.toml looks like:
[python]
enable_resolves = true
resolves_generate_lockfiles = false
[python.resolves]
python-default = "requirements.txt"
I ran ./pants filter --target-type=python_requirement ::
and see
//:reqs#setuptools
//:setuptoolsrefined-addition-53644
12/16/2022, 11:53 AMpython_requirement
isn’t enough if the package which you are using doesn’t define that it has its own dependency on it. So in this case, you need to put an explicit dependency on target associated with setuptools
python_tests(
name="dummy_test.py",
sources=["test_dummy.py"],
dependencies=[":dum", ":reqs#setuptools"],
)
You don’t need to define it again as python_requirement
. Just use the existing one.modern-monkey-78364
12/16/2022, 2:34 PMpython_requirement
with setuptools in project level BUILD file. Only other changes are in pants.toml.
Now the test in first environment is working, whereas the second environment with upgraded pants is failing with below error.
E ModuleNotFoundError: No module named 'pkg_resources'
refined-addition-53644
12/16/2022, 3:35 PMhappy-kitchen-89482
12/16/2022, 3:39 PM//:reqs#setuptools
//:setuptools
[python.resolves]
python-default = "requirements.txt"
requirements.txt
created? Is at an existing file?modern-monkey-78364
12/16/2022, 4:09 PMpoetry export -f requirements.txt -o requirements.txt --without-hashes
python_requirement(
name="setuptools",
requirements=["setuptools"],
)
So, that //:reqs#setuptools is only option.python_requirement
, shouldn't pants install them via pip and use them as dependency? These are the dependencies.
python_requirement(
name="torch",
requirements=["torch==1.10.0+cpu"],
dependencies=[":setuptools"],
)
python_requirement(
name="pytorch-lightning",
requirements=["pytorch-lightning"],
dependencies=[":torch"],
)
And this is the exception it's leading to.happy-kitchen-89482
12/19/2022, 7:21 PMrequirements.txt
as your lockfile. What is the BUILD file target that wraps that? And is pytorch-lightning
in that requirements.txt?requirements.txt
is used to describe the inputs from which a lockfile is generated, not the lockfile itself...modern-monkey-78364
12/19/2022, 7:26 PM[poetry]
version = "poetry==1.3.1"
[python]
interpreter_constraints = ["==3.7.*"]
enable_resolves = true
resolves_generate_lockfiles = false
[python.resolves]
python-default = "requirements.txt"
[pytest]
extra_requirements = ["pytest-mock==3.8.2", "pytest-pgsql==1.1.2", "pytest-icdiff==0.6", "pygments==2.12.0", "pytest-cov==3.0.0", "pytest-bdd==6.0.1"]
lockfile = "pytest_lockfile.txt"
And this is my project level BUILD file.
python_requirement(
name="torch",
requirements=["torch==1.10.0+cpu"],
dependencies=["//:reqs#setuptools"],
)
python_requirement(
name="pytorch-lightning",
requirements=["pytorch-lightning"],
dependencies=[":torch"],
)
python_requirements(
name="reqs",
module_mapping={
"python-constraint": ["constraint"],
"google-api-core": ["google.api_core"],
"grpcio": ["grpc"],
"grpcio-health-checking": ["grpc_health"],
"python-json-logger": ["pythonjsonlogger"],
},
)
requirements.txt
doesn't contain torch or pytorch-lightning, these are only present in BUILD file.happy-kitchen-89482
12/19/2022, 7:31 PMmodern-monkey-78364
12/19/2022, 7:35 PMpytorch-lightning==1.5.0
as a python_requirement in BUILD file is same as adding adding
pytorch-lightning==1.5.0 ; python_version >= "3.7" and python_version < "4.0"
in requirements.txt.
Another question I have is, does pants try to download the requirement mentioned in BUILD file's python_requirement via pip? In above case, should torch and pytorch-lightning be added to requirements.txt to resolve the issue? And if I do add them then is there a use of having them in BUILD file at all?happy-kitchen-89482
12/19/2022, 7:41 PM./pants generate-lockfiles --resolve=python-default
poetry_requirements()
target that wraps your pyproject.tomlpytorch-lightning
and whatever else you're currently manually configuring in BUILD files to your pyproject.tomlmodern-monkey-78364
12/19/2022, 8:14 PM./pants generate-lockfiles
, right?happy-kitchen-89482
12/20/2022, 2:30 AM