brave-hair-40202/12/2023, 11:18 AM
If I was to adopt pants, would there be any need for using namespace packages? Would Pants be able to handle only pulling in the necessary libraries or files?
. ├── libs │ ├── lib1 │ │ ├── company_name │ │ │ └── lib1 │ │ │ ├── __init__.py │ │ │ └── something.py │ │ │ │ │ └── tests │ │ └── test1.py │ └── lib2 │ ├── company_name │ │ └── lib2 │ │ ├── __init__.py │ │ └── something.py │ └── tests │ └── test2.py │ └── executable_apps ├── app1 │ └── app1.py └── app2 └── app2.py
enough-analyst-5443402/12/2023, 1:37 PM
brave-hair-40202/12/2023, 1:49 PM
enough-analyst-5443402/12/2023, 1:54 PM
brave-hair-40202/12/2023, 1:54 PM
enough-analyst-5443402/12/2023, 1:56 PM
brave-hair-40202/12/2023, 2:12 PM
files in subdirectories of namespace packages.
enough-analyst-5443402/12/2023, 9:52 PM
brave-hair-40202/12/2023, 9:55 PM
enough-analyst-5443402/12/2023, 9:56 PM
root patterns. It can't guess either!
does not match what you know they should be, you'll need to configure.
brave-hair-40202/12/2023, 10:02 PM
enough-analyst-5443402/12/2023, 10:06 PM
in a BUILD sibling of the
would do it: https://www.pantsbuild.org/docs/reference-poetry_requirements But one thing you need to shake is the impression that placement implies anything. It does not. If you have 13 pyproject.toml scattered about, each with a BUILD target, then all the deps from all 13 pyproject.toml will be visible to all source files anywhere in the repo. Since Pants parses imports to determine dependencies, this means everything gets exactly what it needs and no more.
What would be a good structure in this case, for making it easy for pants to guess the roots?Unless you're adding new top-level roots at a high clip, structure just doesn't matter. If folks add 1 top-level ~project directory a week, that means adding 1 line to
a week. That is just not too bad! You could change your layout to suit Pants defaults, but it doesn't buy you much.
That said, I have no idea what to do with
[source] roote_patterns = [ "/libs/lib1", "/libs/lib2", ]
. If they are as named, then no other code imports from them and source roots do not matter. If that is not true though, you need to configure for those too.
brave-hair-40202/12/2023, 10:27 PM
, I will use the structure
. The current structure was created to make my packages smaller for building Docker images, but it seems like Pants handles that issue for me.
enough-analyst-5443402/12/2023, 10:29 PM
brave-hair-40202/12/2023, 10:41 PM
but complains about missing dependencies, numpy used in lib1/something.py and hypothesis used in lib1/tests/test1. Both libraries are specified in /libs/lib1/pyproject.toml (numpy under dependencies and hypothesis under dev-dependencies). The pyproject.toml has a sibling BUILD file that looks like this
> pants test libs/lib1::
poetry_requirements( name="poetry", )
enough-analyst-5443402/12/2023, 10:50 PM
and it plopped down some BUILD files with content. Could you share the contents of those if so? Can you point me at a repo or is this all private?
brave-hair-40202/12/2023, 11:01 PM
enough-analyst-5443402/12/2023, 11:01 PM
./pants dependencies --transitive libs/libs1/tests/test1.py
brave-hair-40202/12/2023, 11:05 PM
enough-analyst-5443402/12/2023, 11:07 PM
from numpy import ...
brave-hair-40202/12/2023, 11:13 PM
enough-analyst-5443402/12/2023, 11:15 PM
brave-hair-40202/12/2023, 11:16 PM
enough-analyst-5443402/12/2023, 11:17 PM
brave-hair-40202/12/2023, 11:19 PM
enough-analyst-5443402/12/2023, 11:19 PM
./pants list libs/lib1:
brave-hair-40202/12/2023, 11:21 PM
enough-analyst-5443402/12/2023, 11:21 PM
should have included
./pants dependencies --transitive ...
in the list to satisfy the numpy import.
brave-hair-40202/12/2023, 11:27 PM
enough-analyst-5443402/12/2023, 11:27 PM
brave-hair-40202/12/2023, 11:31 PM
I get dependence on the wrong "pyproject.toml" file from lib2 (although lib1 does not depend on lib2 at all). Same happens if I run
pants dependencies --transitive /libs/lib1/company_name/lib1
pants dependencies --transitive /libs/lib1/tests
enough-analyst-5443402/12/2023, 11:34 PM
- again as a debugging sanity check.
brave-hair-40202/12/2023, 11:40 PM
, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use ..." And then a list of other libraries which use numpy. This might be useful; but I will have to take a better look at this tomorrow. It is getting late her.
enough-analyst-5443402/12/2023, 11:40 PM
brave-hair-40202/12/2023, 11:42 PM
enough-analyst-5443402/12/2023, 11:42 PM
brave-hair-40202/12/2023, 11:43 PM
enough-analyst-5443402/12/2023, 11:44 PM
brave-hair-40202/12/2023, 11:44 PM
enough-analyst-5443402/12/2023, 11:45 PM
adorable-engine-7173602/13/2023, 10:43 AM