Is it possible to set up a pair of Python build targets that are in the same directory level and one of them depends on the other? Specifically, I have a Python project where I want to package all of the

lib

module as a distribution for publishing to PyPI, and I also have code in a sibling module that depends on the contents of

lib

which I want to package separately. The layout looks like:

src/
  concourse/
    BUILD
    lib/
    pipelines/

Hello! Indeed, usually you only want one

python_source

target per file, unless you have conflicting metadata for that same file. Note that

python_sources

is a target generator; it's only for boilerplate reduction to generate those individual

python_source

targets per file. When you do things like package as a distribution, all Pants cares about is the individual targets That means that you have some options: 1. Use only one

python_sources

target generator, and then make your

dependencies

more granular. Right now, it sounds like you're depending on the

python_sources

target, which is an alias for "all of its generated targets". You can instead list the individual generated targets, usually like

src/concourse/lib/utils.py

2. Or, have two

python_sources

targets but update

sources

field so they don't overlap, using

!

globs to exclude possibly

Thanks, I'll take a look at that. Having the separate sources is probably what I'm looking for 🙂 As always your prompt replies are incalculably helpful.

You're welcome! https://www.pantsbuild.org/docs/targets for the relevant docs

So, this is what I'm running into. I've got this in my BUILD file:

python_sources(
    name="concourse-lib",
)
python_sources(
    name="concourse-pipelines",
    sources=["!lib/", "!__init__.py"]
)

python_distribution(
    name="ol-concourse",
    dependencies=[
        ":concourse-lib",
    ],
    description="A Pythonic way to manage your Concourse pipelines",
    provides=python_artifact(
        name="ol-concourse",
        version="0.1.0",
    ),
)

python_distribution(
    name="ol-concourse-pipelines",
    dependencies=[":concourse-pipelines", ":concourse-lib"],
    description="Concourse pipeline definitions used at MIT Open Learning",
    provides=python_artifact(
        name="ol-concourse-pipelines",
        version="0.1.0",
    )
)

I feel like this is similar to this issue. https://pantsbuild.slack.com/archives/C046T6T9U/p1659980088687939

I'm also seeing the

NoSourceRootError

with the following contents of the BUILD file

python_sources(
    name="concourse-lib",
    sources=["lib/**/*.py", "lib/*.py", "__init__.py"]
)

python_distribution(
    name="ol-concourse",
    dependencies=[
        ":concourse-lib",
    ],
    description="A Pythonic way to manage your Concourse pipelines",
    provides=python_artifact(
        name="ol-concourse",
        version="0.1.0",
    ),
)

Not sure if it fixes it, but if you use “lib/**/*.py”, you don’t need “lib/*.py”. So just use the “lib/**/*.py”, if you want package everything inside

lib

as a source

Thanks, I wasn't sure if it would handle the case of direct contents of

lib

. Unfortunately that's still not enough to get rid of the

NoSourceRootError

for

.

I feel like you need to put your python_distribution target just above of

concourse

directory

python_distribution(
    name="ol-concourse",
    dependencies=[
        "./concourse:concourse-lib",
    ],
    description="A Pythonic way to manage your Concourse pipelines",
    provides=python_artifact(
        name="ol-concourse",
        version="0.1.0",
    ),
)

And the BUILD inside the concourse just has the

python_sources(
    name="concourse-lib",
    sources=["lib/**/*.py", __init__.py])

• You can always refer to target in sub directory using relative path from the target definition • For anything else you need to provide the path relative to pants

Thanks, I'll give that a shot...

wait you already have BUILD file inside the

lib

as far as I see the repo

I figured it out. I had to move the

ol-concourse-pipelines

distribution one level deeper under the

pipelines

directory and set the top-level

BUILD

under

src/concourse

to be

python_sources(
    name="concourse",
)

python_distribution(
    name="ol-concourse",
    dependencies=[
        ":concourse",
        "./lib:concourse-libs",
    ],
    description="A Pythonic way to manage your Concourse pipelines",
    provides=python_artifact(
        name="ol-concourse",
        version="0.1.0",
    ),
)