I'm having some trouble understanding how to setup pants. Context: My org's repos have a weird setup where we have 3 (completely python) repos: 1. Foo 2. Bar 3. Baz Now, within our docker images, we mount them like so:

/usr/src/app:Foo
/usr/src/app/bar:Bar
/usr/src/baz:Baz

and then we set the PYTHONPATH within the image to:

/usr/src/app:/usr/src/baz

Now Foo's source root is "/". Bar gets mounted within Foo and so python modules in Foo can import python modules within Bar like so:

import bar.something

Python modules in Foo can also import python modules within Baz like so:

import baz.something

I'm currently keeping

pants.toml

within Foo. Questions: 1. what's the recommended setup here? a. I'm currently thinking of mounting all the above within a container just like our current env and then installing pants within it. 2. should I set the pythonpath env var within the container or given that pants uses its own python binary, should I define it in pants.toml somewhere? a. I do see this link but it refers to plugins which I don't think is right. 3. I'd like to begin by testing pants on a single file/module, let's say located at

Foo/module/submodule

(which can import from across the pythonpath). However setting the source root as "/" and then calling

pants tailor ::

adds BUILD files all over the repo. How can I limit it to only the one submodule? Feel free to have me read/grep documentation if the above is covered somewhere already.

Pants does not use PYTHONPATH nor does it generally allow env variables to influence the processes it runs. As a general concept it is hermetic and tries to run the same regardless of your local env vars vs some other developers. Hopefully you've run down through: https://www.pantsbuild.org/docs/concepts From that list, you want to set up source roots to tell Pants about your 1st party code PYTHONPATH: https://www.pantsbuild.org/docs/source-roots - its the same concept, but language agnostic (CLASSPATH is roughly the mapping for the JVM for example). If you go to the

tailor

docs you'll see a few ignore options: https://www.pantsbuild.org/docs/reference-tailor You might try masking out a subtree in your repo for tailor to operate on using those.

From that list, you want to set up source roots to tell Pants about your 1st party code PYTHONPATH

so the issue is I've defined my pants.toml within Foo and so the only sources I can put are within Foo. However, Baz gets mounted outside the Foo root. Python still treats it as a local module import because of PYTHONPATH

I dare say, IIUC, you are making things incredibly complicated.

haha sadly not my choice, just how the current system is 😞

So, for me to have a hope at understanding I think it's est to not talk about repos. Just the final tree.

sure: / • usr ◦ src ▪︎ app • ... • bar ▪︎ baz

And, for whatever reason, you can't put pants.toml at any of /, /usr or /usr/src - correct?

Ah, I can probably add a step to copy pants.toml into /usr/src

This is pretty crazy. Thinking...

no, some of our dependencies don't like git submodules for whatever reason, hence why we mount them thru docker

Man - I don't know. This is rough.

so the repo mounted at /usr/src/app (Bar) controls the dockerfiles where our images build from pants.toml can probably live next to the dockerfile and hence be version controlled

But just not living where it actually needs to be; thus you need to copy it elsewhere.

yep

I ... this is pretty nuts.

haha im just toying around with pants right now so I'm willing to just use the actual repos to toy around with if it's low effort

This is a totally unique setup in 13 years of Pants to my knowledge; so "low effort" is totally unlikely and unknown.

It does sound like the easiest ways are to either: 1. copy pants.toml to /usr/src so it can see everything 2. change the tree structure so everything lives within /usr/src/app

s/easiest/only/

lol

I can't see other ways.

well I guess I'll see what happens, theoretically this shouldn't be super different from "normal" setups as long as pants sees what it wants to?

Yeah, as long as pants.toml is at or above the root of source files, you're good.

lol we got in this mess due to a factor of airflow and GCP tooling being super new when we adopted them

One thing that will likely be a huge issue is

--changed-since

and anything else that relies on git.

and the easiest way to just get things working was this weird amalgamation of repos

You'll have multiple

.git

dirs IIUC.

ah this is true... the most important repo for changed-since to work on though is /usr/src/app

Yeah, well - I think Pants only looks for

.git

as a sibling of

pants.toml

- I could be remembering that wrong though.

oh oof

You are just in a world of hurt here.

I guess I can always try mounting everything under /usr/src/app

Yeah, exactly.

would git submodules fix this?

I honestly am not sure.

lemme go see if pants has git submodule docs

It definitely does not.

yeah that's fair... ill probably mess around this weekend

Yeah, ok. If you can open source an example setup as you work through issues, that will be infinitely easier to support.

are most people not using git submodules?

I really have no clue. We don't get Slack support questions about them very often if at all that I can recall.

yeah, we have mostly a monorepo except for IAC so that permissions are managed separately and a client config repo so we can push configuration updates to production without having to go through develop and then master

Ok. Sounds complicated!

FYI I did get things seemingly working by mounting everything under /usr/src/app.