https://pantsbuild.org/ logo
Join Slack
Powered by
In our monorepo a developer replaced a mysqlclient...
# general
s
In our monorepo a developer replaced a mysqlclient with psycopg and made some changes in a web app. When I do 
pants list --changed-since=origin/main
it seems to list all of our 3rd party dependencies
Copy code
...
//:reqs#tensorflow
//:reqs#tensorflow-io
//:reqs#typer
...
which is causing our CI to build a lot of unnecessary docker images. Is this expected behavior? Scanning through the git diff I don't see things like tensorflow anywhere
So tbh I don't know how to parse the lock file but if I look at
Copy code
{
          "artifacts": [
            {
              "algorithm": "sha256",
              "hash": "2cd54d7cf979de0e407db182eea27dbed37361af5c329c2dbf71c486be3e432b",
              "url": "<https://files.pythonhosted.org/packages/34/a0/341a28d8a4d638369dd3620c4d356bfc77ceb3450749ca27c805f16a47bf/tensorflow-2.13.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl>"
            },
            {
              "algorithm": "sha256",
              "hash": "3dd6555e9248bace921a311ff05d88239a6b7e96bcd024c5df4c2eaeb1678ac3",
              "url": "<https://files.pythonhosted.org/packages/96/09/10fec9e18e0cc3c2f64596634b4f866a38552eed4ac8a7608d08537e11ca/tensorflow-2.13.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl>"
            },
            {
              "algorithm": "sha256",
              "hash": "17b9a82d69abc487ebad503d61acd11fb24f3b0105be669b5319e55f90f0ca21",
              "url": "<https://files.pythonhosted.org/packages/d4/c9/5d010587ba7792fd0bd12433d0c2b5bc4d59dc7d66d1c1c63b69d77db605/tensorflow-2.13.1-cp310-cp310-macosx_10_15_x86_64.whl>"
            },
            {
              "algorithm": "sha256",
              "hash": "f2935ea4bdf37c1840f4d6ed1ddbb4990dc5ae68e1bc8bba4587bac413195c2f",
              "url": "<https://files.pythonhosted.org/packages/d9/ef/68198a67a487da244c6fdb2433beee39f0a87eea9b4d4491dcb16c0c7747/tensorflow-2.13.1-cp310-cp310-macosx_12_0_arm64.whl>"
            }
          ],
          "project_name": "tensorflow",
          "requires_dists": [
            "absl-py>=1.0.0",
            "astunparse>=1.6.0",
            "flatbuffers>=23.1.21",
            "gast<=0.4.0,>=0.2.1",
            "google-pasta>=0.1.1",
            "grpcio<2.0,>=1.24.3",
            "h5py>=2.9.0",
            "keras<2.14,>=2.13.1",
            "libclang>=13.0.0",
            "numpy<=1.24.3,>=1.22",
            "opt-einsum>=2.3.2",
            "packaging",
            "protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3",
            "setuptools",
            "six>=1.12.0",
            "tensorboard<2.14,>=2.13",
            "tensorflow-cpu-aws==2.13.1; platform_system == \"Linux\" and (platform_machine == \"arm64\" or platform_machine == \"aarch64\")",
            "tensorflow-estimator<2.14,>=2.13.0",
            "tensorflow-intel==2.13.1; platform_system == \"Windows\"",
            "tensorflow-io-gcs-filesystem>=0.23.1; platform_machine != \"arm64\" or platform_system != \"Darwin\"",
            "termcolor>=1.1.0",
            "typing-extensions<4.6.0,>=3.6.6",
            "wrapt>=1.11.0"
          ],
          "requires_python": ">=3.8",
          "version": "2.13.1"
        },
There's no change between the branch and origin/main
I just created a smaller repo with 2 dependencies (requests and tensorflow).
Copy code
.
├── 3rdparty
│   └── python
│       └── default.lock
├── BUILD
├── dist
│   └── export
│       └── python
├── pants.toml
├── requirements.txt
└── src
    ├── ml
    │   ├── BUILD
    │   └── main.py
    └── web
        ├── BUILD
        └── main.py
(edit - web depends only on requests and ml depends on tensorflow) Updating requests I get
Copy code
╰❱ pants dependents --transitive --changed-since=master
//:root
//:root#requests
//:root#tensorflow
3rdparty/python:_python-default_lockfile
src/ml:ml
src/ml/main.py
src/web:web
src/web/main.py
How do folks deal with this in larger projects?
c
when you touch the requirements file, it invalidates all of them as pants doesn’t have a connection between the lines in your requirements and the corresponding 
python_requirement
target.
in larger projects, you either take the hit that when a requirement is changed, it invalidates the world, or break your repo down into smaller parts with one resolve per part.
👍 1
5 Views
Open in Slack
PreviousNext
Powered by