https://pantsbuild.org/ logo
Join Slack
Powered by
Hello :wave: First of all, thanks for building thi...
# general
b
Hello šŸ‘‹ First of all, thanks for building this amazing build system šŸ™‡ I am building a Python monorepo that contains multiple 
python_distribution
targets. When I package any of the distributions, it gets the correct dependencies versions (third-part and other distributions in the same repo). However, when I run tests for a specific distribution, I see that the venv created has the dependency version of the other distributions as the latest stable versions published previously and not the new version that current defined in other distribution BUILD files. Is that by design? Am I doing something wrong? pants.toml:
Copy code
[GLOBAL]
pants_version = "2.16.0"
colors = true
dynamic_ui = true
backend_packages = [
  "pants.backend.python",
  "pants.backend.python.typecheck.mypy",
  "pants.backend.python.lint.isort",
  "pants.backend.python.lint.autoflake",
  "pants.backend.python.lint.black",
  "pants.backend.experimental.python.lint.ruff",
  "pants.backend.python.lint.pylint",
]
plugins = ["hdrhistogram"]

[export]
resolve = ["python-default"]
py_resolve_format = "mutable_virtualenv"

[python]
interpreter_constraints = ["CPython==3.9.*"]
enable_resolves = true
tailor_ignore_empty_init_files = false

[python.resolves]
python-default = "3rdparty/python/default.lock"

[generate-lockfiles]
diff = true

[python-infer]
init_files = "always"

[python-bootstrap]
search_path = ["<PATH>", "<PYENV>"]

[setup-py-generation]
first_party_dependency_version_scheme = "compatible"

[test]
report = true
report_dir = "dist/test_results"
use_coverage = true

[coverage-py]
global_report = true
interpreter_constraints = ["==3.9.*"]
report = ["xml"]
output_dir = "dist/coverage"

[pylint]
args = ["--disable=C0103,C0301,E0401,E0611,R0903,R0801,R1735"]
install_from_resolve = "python-default"

[mypy]
install_from_resolve = "python-default"

[pytest]
args = ["--cache-clear"]
install_from_resolve = "python-default"

[anonymous-telemetry]
enabled = false
āœ… 1
āœ… 1
h
Tests should not rely on published distributions from the repo at all. They should run entirely at HEAD.
Can you put up a small example repo that demonstrates what you're seeing?
Do you have your own (first party) dists in your requirements.txt or something? You should not typically want to do that
b
I have created a test repo to elaborate what am I trying to achieve. https://github.com/khadrawy/platform-test
Do you have your own (first party) dists in your requirements.txt or something?
No, I have only third party deps in requirements.txt
e
@better-leather-15054 can you also add a README with the command to run and then what to observe as wrong after vs what you expected?
b
e
Ok, great. Thanks. I'll take a look.
There is still no actual command run, just prose. Is 
pants test ::
OK to use for repro?
Ok, to get this to work you 1st need:
Copy code
$ git diff
diff --git a/pants.toml b/pants.toml
index 40bc29b..548f790 100644
--- a/pants.toml
+++ b/pants.toml
@@ -57,7 +57,7 @@ install_from_resolve = "python-default"

 [pytest]
 args = ["--cache-clear"]
-install_from_resolve = "python-default"
+#install_from_resolve = "python-default"

 [anonymous-telemetry]
 enabled = false
Then get:
Copy code
$ pants test ::
07:52:49.01 [INFO] Completed: Building pytest.pex from <resource://pants.backend.python.subsystems/pytest.lock>
07:52:49.75 [INFO] Completed: Building local_dists.pex
07:52:50.71 [INFO] Completed: Building pytest_runner.pex
07:52:51.09 [INFO] Completed: Run Pytest - repositories/platform/tests/test_dummy.py:tests_platform - succeeded.
07:52:51.09 [INFO] Completed: Run Pytest - repositories/consumer/tests/test_consumer.py:tests_consumer - succeeded.

āœ“ repositories/consumer/tests/test_consumer.py:tests_consumer succeeded in 0.37s.
āœ“ repositories/platform/tests/test_dummy.py:tests_platform succeeded in 0.37s.

Wrote test reports to dist/test_results
07:52:51.99 [INFO] Completed: Building coverage_py.pex from <resource://pants.backend.python.subsystems/coverage_py.lock>

Wrote xml coverage report to `dist/coverage`
So this repo is maybe useful to help understand your problem but not to actually repro it. Presumably you have the real repo pointed at some custom PyPI index that has published versions of 
consumer
, etc?
Aha - yes: https://pip-artifacts.vrff.io/. So the thing to focus on here @better-leather-15054 is what are the differences between the real case and the repo you exposed for us to test. It's one or more of those differences that is the key issue here.
b
@enough-analyst-54434 I think I found the issue, which is on my side. As I am migrating packages one by one to the monorepo, there is still a common dependency between the two packages outside which depends also on the consumer. So:
Copy code
platform -depends-> consumer
platform -depends-> aggregator (still outside monorepo)
aggregator -depends-> consumer
Now it makes sense to see that venv for the test has v6.0.0. I was distracted by the fact that when I package 
platform
I get 
setup.py
that indicates the it depends on 
consumer~=7.0.0
which has nothing with building the pex for test. Thanks a lot for your help šŸ™‡
e
Aha. Ok, great. You're welcome.
Open in Slack
PreviousNext
Powered by