cold-sugar-54376
05/28/2022, 2:34 AMpyproject.toml
and poetry_requirements
to define requirements for my pants project. I’m finding that everytime I change/upgrade a dependency (even a dev one such as pytest
) in pyrpoject.toml
pants tries to rebuild all my targets (even pex_binaries which don’t depend on it). Any idea why this could be happening? From what I can gather it’s because pyproject.toml ends up being a transitive dependency of every target in the repoenough-analyst-54434
05/28/2022, 3:12 AMcold-sugar-54376
05/28/2022, 3:37 AMpython_requirement(
name="setuptools",
requirements=["setuptools"],
)
poetry_requirements(
module_mapping={
"django-admin-sortable2": ["adminsortable2"],
"django-admin-sso": ["admin_sso"],
"django-anymail": ["anymail"],
"django-environ": ["environ"],
"django-import-export": ["import_export"],
"django-inline-actions": ["inline_actions"],
"django-ipware": ["ipware"],
"django-safedelete": ["safedelete"],
"drf-nested-routers": ["rest_framework_nested"],
"google-api-python-client": ["googleapiclient"],
"ShopifyAPI": ["shopify"],
},
)
./pants dependencies --transitive src/util/service.py
3rdparty#gunicorn
3rdparty/pyproject.toml
src/util/__init__.py
enough-analyst-54434
05/28/2022, 3:45 PMcold-sugar-54376
05/28/2022, 5:17 PMpytest
version, pants would be smart enough to know if I did ./pants --changed-since=asdf --changed-dependees=transitive filter --target-type=docker_image
that i don’t actually need to rebuild the docker imageenough-analyst-54434
05/28/2022, 5:30 PMpackage
we always run that last side-effecting step where we create the package. All or most of the internal steps needed to prepare that final side-effect should be skipped / read from cache.cold-sugar-54376
05/28/2022, 5:34 PMpackage
based on a filter
./pants --changed-since HEAD --changed-dependees=direct list
I get a list of every 3rdparty#<dep>
target instead of just 3rdparty#pytest
transitive
and pipe that into a filter for docker images to see what actually needs to be built. I took it from this blog post: https://blog.pantsbuild.org/pants-pex-and-docker/./pants --changed-since=main --changed-dependees=transitive filter \
--target-type=docker_image | \
xargs ./pants package
enough-analyst-54434
05/28/2022, 5:38 PMlist
. At runtime, when executing rules and processes to achieve goals, Pants looks at the inputs to the rule or process and if they have not changed, it short circuits at that point../pants --changed-since=main --changed-dependees=transitive package
?cold-sugar-54376
05/28/2022, 5:42 PMenough-analyst-54434
05/28/2022, 5:43 PMcold-sugar-54376
05/28/2022, 5:44 PMenough-analyst-54434
05/28/2022, 5:45 PM11:44:37.60 [INFO] Initializing scheduler...
11:44:38.00 [INFO] Scheduler initialized.
As the 1st 2 Pants output lines or are you using --no-pantsd?cold-sugar-54376
05/28/2022, 5:46 PMenough-analyst-54434
05/28/2022, 5:46 PMpants could say “oh ok only this docker image needs to be built ...That much we should be getting right currently mod speed. The docker image might "rebuild" but as a noop.
cold-sugar-54376
05/28/2022, 5:48 PMenough-analyst-54434
05/28/2022, 5:48 PMcold-sugar-54376
05/28/2022, 5:48 PMenough-analyst-54434
05/28/2022, 5:49 PMcold-sugar-54376
05/28/2022, 5:49 PMenough-analyst-54434
05/28/2022, 5:50 PMcold-sugar-54376
05/28/2022, 5:50 PMenough-analyst-54434
05/28/2022, 5:59 PMcold-sugar-54376
05/28/2022, 6:00 PMlist
or filter
to have it handle dependency file updates in the way i described?enough-analyst-54434
05/28/2022, 6:01 PMis there a way we could add an option further down into something likeHrm, I'm not sure how bad that would be. This is definitely not awesome:orlist
to have it handle dependency file updates in the way i described?filter
$ git diff
diff --git a/3rdparty/python/requirements.txt b/3rdparty/python/requirements.txt
index 551931f1e..0e7e39f0b 100644
--- a/3rdparty/python/requirements.txt
+++ b/3rdparty/python/requirements.txt
@@ -4,6 +4,8 @@
# Additionally, it increases the surface area of Pants's supply chain for security.
# Consider pinging us on Slack if you're thinking a new dependency might be needed.
+cowsay
+
ansicolors==1.1.8
chevron==0.14.0 # Should only be used by build-support.
fasteners==0.16.3
$ ./pants --changed-since=HEAD --changed-dependees=transitive filter --target-type=pex_binary
build-support/bin#_generate_all_lockfiles_helper.py
build-support/bin#_release_helper.py
build-support/bin#changelog.py
build-support/bin#generate_docs.py
build-support/bin#generate_github_workflows.py
build-support/bin#generate_user_list.py
build-support/bin#reversion.py
src/python/pants/bin:pants
testprojects/src/python/hello/main:main
testprojects/src/python/native:main
pex_binary
targets changed, which, as you point out, is totally untrue.cold-sugar-54376
05/28/2022, 6:16 PMenough-analyst-54434
05/28/2022, 6:27 PMcold-sugar-54376
05/28/2022, 6:32 PM