gentle-tailor-79858
05/15/2023, 2:45 PMincremental adoption
method to get pants into one of our internal repositories. @hundreds-father-404 was kind enough to get me started with an example plugin. I'm moving my questions here though to not bog them down too much. The example plugin exists in its own repo, and I have a seperate target repo I'd like to start integrating pants into, I'm thinking I start a new branch in the target repo and add the pants.toml, but I've got a couple questions
1. When I set up my pants.toml and spell out the new plugin, is pants "system aware" of my plugin? Do I need to add the plugin to a "pants environment" or something to that effect?
2. When I run pants the first time in a repo it initializes some things in the project right? Is there anything that gets generated I need to commit.
I'm thinking I should just try this but I'm so new to pants I don't want to break anything about the "pants install". I assume that's not a problem really but better safe to ask first 🙂happy-kitchen-89482
05/15/2023, 2:54 PMpants tailor
(after reviewing them for correctness) but not the results of just pants
.pants --version
, say, should bootstrap pants, but shouldn't create any non-gitignored filesgentle-tailor-79858
05/15/2023, 2:57 PMIf the plugin is in another repo then presumably you're publishing it somewhere to be consumed by the target repo?Correct, the separate repo was for some basic testing to make sure things were suitable. And we'd like to reuse the function on multiple target repos. So my question is as you stated: How does the target repo consume the plugins/become aware of them?
pythonpath = ["<path/to/local/repo>/pants-plugins"]
enough-analyst-54434
05/15/2023, 3:10 PMpants list ::
.happy-kitchen-89482
05/15/2023, 3:16 PM[GLOBAL].plugins
option in pants.toml:
[GLOBAL]
plugins = [
"my.plugin==1.2.3",
]
register.py
and invoke itgentle-tailor-79858
05/15/2023, 3:42 PMpants list ::
is complaining that it's failing to read a link, but I can list that link just fine. I think the problem might be the file system that it's pointing to with that link is extremely slow. The location of that real file is on a server across the country from me.pants tailor ::
IntrinsicError: Failed to read link "/path/to/symlink": Absolute symlink: "/the/real/path"
To be clear I don't expect pants to be super helpful out of the gate. It's not a python based repo and that's the purpose of the language plugin. So this probably doesn't actually matter for now as long as I can get the language-plugin goals to start working.enough-analyst-54434
05/15/2023, 4:07 PMgentle-tailor-79858
05/15/2023, 4:08 PMenough-analyst-54434
05/15/2023, 4:09 PMgentle-tailor-79858
05/15/2023, 4:10 PMInternal PyPI is the primary use case, along with in-repo plugins. Sibling repo plugins obviously works but is decidedly not the norm.OH, WAIT DUH! I could just use a git submodule right?
pythonpath = ["%(buildroot)s/pants-plugins"]
enough-analyst-54434
05/15/2023, 4:12 PMhappy-kitchen-89482
05/15/2023, 4:13 PMenough-analyst-54434
05/15/2023, 4:14 PMgentle-tailor-79858
05/15/2023, 4:15 PMenough-analyst-54434
05/15/2023, 4:45 PM--changed-since
, .gitignore
and I think that's about it.gentle-tailor-79858
05/15/2023, 4:50 PMenough-analyst-54434
05/15/2023, 4:50 PMensure-sane-setup
).gentle-tailor-79858
05/15/2023, 4:51 PMenough-analyst-54434
05/15/2023, 4:51 PMgentle-tailor-79858
05/15/2023, 4:52 PMI assume folks who are in your situation generally don't try to shoehorn that management into pants. They use some other tool (say:The first goal is).ensure-sane-setup
check-env
😅enough-analyst-54434
05/15/2023, 4:52 PMgentle-tailor-79858
05/15/2023, 4:52 PMWhy not host the plugin in-repo?There's a division of permissions in the organization
enough-analyst-54434
05/15/2023, 4:52 PMhundreds-father-404
05/15/2023, 4:53 PMI'm moving my questions here though to not bog them down too much.No worries about that, feel free to ask me questions directly 🙂
Why not host the plugin in-repo?One reason I helped write the plugin but don't have auth for Austin's hardware repository, and it would be a big production to get it. The idea was to help write the generic code in a repo, then copy and paste that into the dedicated single repository. If multiple repos ended up needing it, then consider publishing
gentle-tailor-79858
05/15/2023, 4:54 PMhundreds-father-404
05/15/2023, 4:54 PMI question the wisdom of that being a Pants goal is what I'm saying insofar as it deals with things outside the repo.The idea is that the underlying tools like compilers will access the absolute paths. Have a goal to make sure it's set up. And that goal's rule will run automatically when doing things like
check
and lint
Indeed, Pants's file watcher and LMDB caching would not be able to cover changes to those absolute file paths.enough-analyst-54434
05/15/2023, 4:55 PMhundreds-father-404
05/15/2023, 4:56 PMenough-analyst-54434
05/15/2023, 4:56 PMgentle-tailor-79858
05/15/2023, 4:56 PMenough-analyst-54434
05/15/2023, 4:56 PMgentle-tailor-79858
05/15/2023, 4:56 PMhundreds-father-404
05/15/2023, 4:57 PMJust trying to fix itJosh Cannon's Pycon talk helped inspire Austin to want to make life better. It won't ever get as nice as we're spoiled with, but a 20% improvement is substantial given how low the bar is etc
gentle-tailor-79858
05/15/2023, 4:57 PMenough-analyst-54434
05/15/2023, 4:57 PMgentle-tailor-79858
05/15/2023, 4:59 PMhappy-kitchen-89482
05/15/2023, 5:02 PMhundreds-father-404
05/15/2023, 5:03 PMSo it sounds like you can use the Pants engine as the basis of your own custom build, basicallyPrecisely. This is the plan. Rather than trying to design a custom build system or using Bazel Also, Austin and I think dep inference will be super valuable. The dep rules are hard for humans to understand, it sounds like, but fine for machines
gentle-tailor-79858
05/15/2023, 5:12 PMSo it sounds like you can use the Pants engine as the basis of your own custom build, basically💯💯💯 Since Pants in it's purest form is a build system / coordinator, it's perfect for handling these esoteric use cases.
Also, Austin and I think dep inference will be super valuable. The dep rules are hard for humans to understand, it sounds like, but fine for machinesOh absolutely, this is the part that I would even like to open source. I spent a lot of time thinking about how to handle this and seeing what's out there, unfortunately there's not a lot, this is one of the better toolkits I've found over the years. It makes calls to dependencies that would be good tools for handling dep inference I think https://terostechnology.github.io/terosHDLdoc/