Hey again! It's been a while. I wanted to check in...
# general
f
Hey again! It's been a while. I wanted to check in and see if anyone has been using pants solely for its dependency inference and build graph mapping capabilities; i.e., just using
./pants list
and friends as ways to generate input for other build tools? I ask this because I'm looking for a way to introduce this at my new org, but there's a lot of build work already done that is deeply incompatible with the pants way of doing things, and just trying to get that all working in pants' model on the side would be a big, high-risk project. It seems more sane to try to introduce it slowly.
h
Hey Josh, good to see you on here again!
Funny you should ask, we have been having quite a few discussions along those lines
i.e., how to use Pants in helpful ways that are additive to what already exists in a repo
We don't want the perception to be that for Pants to be useful you first have to do a big migration, or refactoring, or replace existing idioms and workflows
👍 1
we want Pants to support existing workflows, adding automation, caching, insights etc
and a big part of that is indeed the build graph capabilities, and how that gives better performance to tools and better insight to humans
So basically if there is a "pants way of doing things" then that is bad and we should make pants work with your existing build workflows
Could you describe those a bit more?
f
Well the pants way of doing things isn't bad in and of itself, (in fact I think it's good), but it does assume that build steps should be side-effect free, cacheable, and stock python goals are deeply tied to pex
I'm working now with a monorepo whose build system consists of various makefiles and support scripts around those, and almost all our artifacts are RPMs. I think unlike most orgs, instead of splitting a repo when it got too big to understand or using tooling that explicitly models dependencies within, we've just kicked that can down the road by using kitchen sink packaging and try to pretend dependencies don't matter. What we're left with is an enormous single build an single deployment for a sprawling multi-component system that's really hard to reason about and causes a lot of problems. I call this the "monobuild."
But anyways, we're not even using some pretty common things like standard python packaging, instead we're fully bought-in to RPMs, to the point where we repackage some upstream PyPI stuff as RPMs in our own repo. Writing pants extensions do this stuff isn't impossible, but it will take time, and I won't get that time just by saying "no, for real folks, I swear it works."
👍 1
My idea is to start using pants just for dependency modeling, and avoiding it's more active build goals for the time being. In this way, I'd be able to ask it questions like "what test files do I need to run for this changeset"? And then run those tests not in pants, but pass that information on to some other test runner that doesn't need to run things in a hermetically sealed environment.
👍 1
h
Makes sense, I actually like that as a low-pressure way to introduce the tool and then go from there
1
What would kind of features would you like to see that aren't already present?
I could imagine a "run this shell script on relevant files but never cache its results" kind of thing?
h
Yeah that sounds great. I imagine there is work to be done to improve the project introspection goals, like forcing it to output literal file names rather than target addresses (including "file targets" which often are the same as file names)
f
It's gonna be hard to say what features I need until I start digging in to actually use it in this repo. Things I could say now like "improve project introspection goals" or "decouple from pex more" are too high-level to be actionable. I'll report back if and when I get to diving into this more.
2
h
I can imagine Pants being useful in "run directly in the repo, not in a sandbox, I accept responsibility for side-effects" mode. But then we wouldn't do as much caching.