Is there a TLDR on settings `pants.toml` for `buil...
# general
w
Is there a TLDR on settings
pants.toml
for
build_patterns
marker_filenames
and
root_patterns
? I think
marker_filenames
and
root_patterns
both set the source root, via different mechanisms, and then
build_patterns
is what Pants looks for to identify something pants-able. However, I just did a re-factor, and I'm not sure why I lost access to one of my libraries. /core/BUILD.pants
Copy code
python_sources(
    name="libcore",
    sources=["**/*.py"],
)
/helloworld/BUILD.pants
Copy code
python_sources(
    name="libhelloworld",
    sources=["**/*.py"],
    dependencies=[
        "core:libcore",
    ],
)
This above, works, however, nested everything under
examples/python
(without adding new BUILD files) and it falls over. My
pants.toml
is as follows:
Copy code
[GLOBAL]
pants_version = "2.11.0.dev2"
build_patterns = ["BUILD.pants", "BUILD"]

[source]
marker_filenames = ["BUILD.pants"]
root_patterns = [
    "examples/python",
    "pants-plugins",
]

...
After re-reading https://www.pantsbuild.org/docs/existing-repositories#migrating-from-other-build-tools-set-custom-build-file-names - I think I can drop the
build_patterns
entirely, and while I can add another top-level BUILD to the
examples/python
directory, I'm not sure why it worked top level but then fails nested... Note: My
pants.toml
had "/" as a root_pattern, when everything was top-level
Note: I know that I can fully qualify those core libs to
examples/python/core:libcore
- but I thought one of the
build_patterns
or
source_roots
would have solved that problem
Also, I've seen this: https://www.pantsbuild.org/docs/targets#target-generation This makes me think the only options are fully-qualified, or relative. I'm not sure where I got it in my head that relative to the source root was a thing, but I would have sworn that should've worked before I did it and failed
s
@wide-midnight-78598 2 things: • Is there a specific error you are encountering? Posting the log from the error might expose what went wrong or help understand your problem. • This is just a guess, but have you tried playing around with listing the targets with specific settings?
./pants list ::
and
./pants roots
helped me a lot to understand what happens when I change my settings.
w
Thanks @strong-toothbrush-37759 - no there is no specific error and I've tested out the
./pants list ::
and
./pants roots
(but I honestly forget what those API calls are EVERY TIME I'm debugging something. It's the strangest thing. Anyways, it's not an error problem, as much, as a qualitative API "surprise"? That's not even the correct word, but I can fully qualify everything, and it works correctly - but it definitely looks ugly in the build files.
w
BUILD file names are generally independent from source roots (which are defined by the
[sources]
subsystem): source roots are how filenames get trimmed at import time: see https://www.pantsbuild.org/docs/source-roots for more info.
you usually do not want to define source roots based on where BUILD files are, since BUILD files should be created near the code they describe (and adding and removing them should not affect your import paths!). generally, you will want to use
./pants tailor
to create BUILD files rather than creating them manually. see https://www.pantsbuild.org/v2.10/docs/create-initial-build-files for more info.
w
Thanks @witty-crayon-22786 - so I guess the question is, is it possible to do what I'm trying to do? Have a directory a couple folders deep, with sibling projects accessing a common library, without fully-qualifying the core lib in the BUILD files?
s
If those files in deep nesting try to access
from libcore import something
it should generally be possible since Pants should infer the dependency. But again as @witty-crayon-22786 mentioned, you should probably use
./pants tailor
to create your BUILD files rather than trying to manually create them yourself.
w
@wide-midnight-78598: re:
Note: I know that I can fully qualify those core libs to
examples/python/core:libcore
- but I thought one of the
build_patterns
or
source_roots
would have solved that problem
no: neither of these allow for shorthand when you explicitly reference those libraries in a
dependencies
list. but as @strong-toothbrush-37759 said: needing to explicitly add
dependencies
lists should be rare due to dependency inference. if you are needing to add a dep manually, it might be because your source roots aren’t set up properly.
🙌 1
the connection between source roots and dependency inference is that the source root “chops off” the filesystem prefix that isn’t relevant to the name of your python module. the “right amount” needs to be chopped off of
examples/python/core
to determine that the module name is
core
and (and so should be imported as
from core import something
) : and that means that you need
./pants roots
to report a source root at
examples/python
w
Thanks @witty-crayon-22786! That's kinda what I figured in the end. I wasn't sure if it was language-only, or language + Pants setup. I only need explicit dependencies occasionally, due to some weird naming and re-aliasing in a few projects. However, in those projects, it ends up being weird to have -some- dependencies specified, but the rest inferred. Doesn't leave me with a good stylistic taste in my mouth. The other value in specifying them explicitly is migrating clients or other projects over. More often than not, the people I work with (and frankly, myself-included), prefer explicit over implicit (zen of python style, sorta, but other languages too). Moreso for me, the "just works like magic" burns me as often as it helps me - so I've tended to lean more explicit in my setup and build files over the years. Not saying it's a good habit, but it's worked out better on average in my use cases. It's also why I even specify some important target fields explicitly, even if they match default - because I can't guarantee that every default will remain the same over the life of my project(s). Anyways, this specific use case I asked about - I don't mind using Pant's dep inference, since it's only for new pants-plugins I'm workshopping and it's an intentionally awkward folder structure - since I'm covering independent examples in multiple languages
w
Yeah: the cases in which you need explicit dependencies, are actually exactly the ones where specifying only the package wouldn't be specific enough, and you actually need to fully specify the target (ambiguity, etc)
👍 1