I'll try to explain what I expect to do with Pants, as I am having some issues in getting started (but maybe my approach is the issue). We are moving a bunch of python (and cython) project to a monorepo structure, I read some articles about different approaches and the way we want go for involves having multiple "project roots" in subfolders. Each project root has its own pyproject.toml, poetry.lock and set of dependencies. Does pants support this setup? I couldn't find any examples or documentation to answer this question, maybe I'm just blind though 😄. I tried this very quickly, so maybe I'm missing some obvious stuff,

./pants dependencies projects/example

doesn't return any dependency, I tried both

poetry_requirements

and

python_requirements

. If pants can indeed support this setup, do you have any examples of open source monorepo?

hey @plain-fireman-49959! welcome.

Yes, I did run

./pants roots

and I get all the roots I expect (some are "double nested" like

one.two.project_1

,

one.two.project_2

etc)

hm… not sure what you mean by double nested. but the longest root will match: so if you see an inaccurate nested root, you’d want to adjust your markers

ok thank you. the roots look right to me, I'll setup a test project so I don't leak company stuff and i'll try this

what does

./pants roots

report in this case?

I opened the root in vscode and it resolves

one.b

as I'd like it to.

all of the roots below

one

are problematic

a, b and c folder are project roots, each one has a pyproject.toml etc

because the effect of

one/a

is that

one/a

will be stripped from the import path

I had

marker_filenames = ["pyproject.toml"]

in

pants.toml

(removed it for this screenshot)

@plain-fireman-49959: i think that maybe you’re halfway between “Multiple top-level projects” and the monorepo traditional

src/<lang>

layout

oh ok

and can 

a

 be a project with its own pyproject and set of deps?

mostly: when dependencies overlap between projects, Pants will report them as “ambiguous”, and you’ll need to declare them in

BUILD

files. so it will work, but IMO it’s more of a transitional state on the way to having fewer sets of thirdparty deps declared, and only when different parts of the repository need different deps

ok I think I am somewhat confused: I imagine that by discovering dependencies by looking at imports as pants does, I can have a "root" pyproject and run goals for each subproject by "downloading" only its set of dependencies, I can package and distribute them independently with only their dependencies. The main problem I'm having though is that we mostly run all this project from one monolithic entry point/service. With the multi-repo setup we currently have we have to deploy new versions of the monolith everytime we release a dependency.. I know this can be managed as a huge standalone project but we like the idea of running specific developer workflows quickly for each subproject and have the option of releasing independent version if we need to use them in random microservices

I imagine that by discovering dependencies by looking at imports as pants does, I can have a “root” pyproject and run goals for each subproject by “downloading” only its set of dependencies, I can package and distribute them independently with only their dependencies. The main problem I’m having though is that we mostly run all this project from one monolithic entry point/service.

yea, all of this is correct. you will be able to run tests using only the relevant subset of the dependencies, and only when those dependencies have changed. it’s fine to then have a smaller number of

pex_binary

targets: i.e., things that are actually deployed

yes packaging and distributing would be a small and infrequent use case for us!

sure thing!

To follow up on Stu's comments about third-party deps and multiple

pyproject.toml

, check out https://www.pantsbuild.org/docs/python-third-party-dependencies for Pants's conceptual model of third-party dependencies and how it creates a single "universe" of dependencies that any file can pull from

mm, i also realized that i missed that your layout was

one/a/a/a.py

… if you want to import as

from a.a import A

, then you would need a source root at

one/a

(

one/a/a/a.py

has the source root chopped off to get

a/a.py

, which would be imported as

from a import a

or

from a.a import A

). but anyway: just think of the source roots as the prefix to chop off before calculating the import path.

@hundreds-father-404 yes I had a good look at the part of the docs, I now understand why you would want to define "globally", thanks!

Great! If you haven't already, we recommend running ./pants tailor to generate those BUILD files