brave-hair-402
02/12/2023, 11:18 AM.
├── libs
│ ├── lib1
│ │ ├── company_name
│ │ │ └── lib1
│ │ │ ├── __init__.py
│ │ │ └── something.py
│ │ │
│ │ └── tests
│ │ └── test1.py
│ └── lib2
│ ├── company_name
│ │ └── lib2
│ │ ├── __init__.py
│ │ └── something.py
│ └── tests
│ └── test2.py
│
└── executable_apps
├── app1
│ └── app1.py
└── app2
└── app2.py
If I was to adopt pants, would there be any need for using namespace packages? Would Pants be able to handle only pulling in the necessary libraries or files?enough-analyst-54434
02/12/2023, 1:37 PMbrave-hair-402
02/12/2023, 1:49 PMenough-analyst-54434
02/12/2023, 1:54 PMbrave-hair-402
02/12/2023, 1:54 PMenough-analyst-54434
02/12/2023, 1:56 PMcompany_name
?__init__.py
it seems.brave-hair-402
02/12/2023, 2:12 PM__init__.py
files.__init__.py
files in subdirectories of namespace packages.enough-analyst-54434
02/12/2023, 9:52 PMbrave-hair-402
02/12/2023, 9:55 PMenough-analyst-54434
02/12/2023, 9:56 PM[source]
root patterns. It can't guess either!./pants roots
does not match what you know they should be, you'll need to configure.brave-hair-402
02/12/2023, 10:02 PMenough-analyst-54434
02/12/2023, 10:06 PMpoetry_requirements()
in a BUILD sibling of the pyproject.toml
would do it: https://www.pantsbuild.org/docs/reference-poetry_requirements
But one thing you need to shake is the impression that placement implies anything. It does not. If you have 13 pyproject.toml scattered about, each with a BUILD target, then all the deps from all 13 pyproject.toml will be visible to all source files anywhere in the repo. Since Pants parses imports to determine dependencies, this means everything gets exactly what it needs and no more.What would be a good structure in this case, for making it easy for pants to guess the roots?Unless you're adding new top-level roots at a high clip, structure just doesn't matter. If folks add 1 top-level ~project directory a week, that means adding 1 line to
root_pattern
a week. That is just not too bad! You could change your layout to suit Pants defaults, but it doesn't buy you much../pants tailor
? After configuring source roots you should. It auto-generates BUILD targets including the poetry_requirements
ones. https://www.pantsbuild.org/docs/initial-configuration[source]
roote_patterns = [
"/libs/lib1",
"/libs/lib2",
]
That said, I have no idea what to do with executable_apps
. If they are as named, then no other code imports from them and source roots do not matter. If that is not true though, you need to configure for those too.brave-hair-402
02/12/2023, 10:27 PM/libs/lib1/company_name/lib1
, I will use the structure /libs/company_name/lib1
. The current structure was created to make my packages smaller for building Docker images, but it seems like Pants handles that issue for me.enough-analyst-54434
02/12/2023, 10:29 PMbrave-hair-402
02/12/2023, 10:41 PM> pants test libs/lib1::
but complains about missing dependencies, numpy used in lib1/something.py and hypothesis used in lib1/tests/test1.
Both libraries are specified in /libs/lib1/pyproject.toml (numpy under dependencies and hypothesis under dev-dependencies).
The pyproject.toml has a sibling BUILD file that looks like this
poetry_requirements(
name="poetry",
)
Any ideas?enough-analyst-54434
02/12/2023, 10:50 PM__import("...")__
?./pants tailor
and it plopped down some BUILD files with content. Could you share the contents of those if so? Can you point me at a repo or is this all private?brave-hair-402
02/12/2023, 11:01 PMpython_sources()
and /libs/lib1/tests/BUILD/
python_tests()
enough-analyst-54434
02/12/2023, 11:01 PM./pants dependencies --transitive libs/libs1/tests/test1.py
show?brave-hair-402
02/12/2023, 11:05 PMlibs/lib2/pyproject.toml:poetry
libs/lib1/pyproject.toml:poetry
enough-analyst-54434
02/12/2023, 11:07 PMimport numpy
or from numpy import ...
in it?brave-hair-402
02/12/2023, 11:13 PMenough-analyst-54434
02/12/2023, 11:15 PMbrave-hair-402
02/12/2023, 11:16 PMenough-analyst-54434
02/12/2023, 11:17 PMbrave-hair-402
02/12/2023, 11:19 PMenough-analyst-54434
02/12/2023, 11:19 PM./pants list libs/lib1:
brave-hair-402
02/12/2023, 11:21 PMlibs/lib1:poetry#hypothesis
libs/lib1:poetry#numpy
enough-analyst-54434
02/12/2023, 11:21 PM./pants dependencies --transitive ...
should have included libs/lib1:poetry#numpy
in the list to satisfy the numpy import.brave-hair-402
02/12/2023, 11:27 PMenough-analyst-54434
02/12/2023, 11:27 PMbrave-hair-402
02/12/2023, 11:31 PMpants dependencies --transitive /libs/lib1/company_name/lib1
I get dependence on the wrong "pyproject.toml" file from lib2 (although lib1 does not depend on lib2 at all). Same happens if I run
pants dependencies --transitive /libs/lib1/tests
enough-analyst-54434
02/12/2023, 11:34 PM--no-pantsd
- again as a debugging sanity check.brave-hair-402
02/12/2023, 11:40 PMnumpy
, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use ..."
And then a list of other libraries which use numpy. This might be useful; but I will have to take a better look at this tomorrow. It is getting late her.enough-analyst-54434
02/12/2023, 11:40 PMbrave-hair-402
02/12/2023, 11:42 PMenough-analyst-54434
02/12/2023, 11:42 PMbrave-hair-402
02/12/2023, 11:43 PMenough-analyst-54434
02/12/2023, 11:44 PMbrave-hair-402
02/12/2023, 11:44 PMenough-analyst-54434
02/12/2023, 11:45 PMadorable-engine-71736
02/13/2023, 10:43 AM