Hey all, seeing a bit of chatter about "pants v2"....
# general
Hey all, seeing a bit of chatter about "pants v2". I'm trying to get a new project set up, is this still way too early for looking at that? Not seeing any info online about this
Hello! Are you using Python or another language? If python, we recommend starting with V2 by using a rewrite of the docs at https://pants.readme.io/docs/welcome-to-pants (some parts are still a work in progress) V2 is generally a better and more performance experience, outside of a few areas where we’re applying polish.
hmm yeah that looks good. My wrinkle is I also have some stuff running through grunt/webpack. I haven't quite found a good equivalent to Bazel's
(is it
?), I guess I can write up a plugin for this pretty easily though
Are you using Python, JavaScript, or both?
Both. Basically a very large Django application with a frontend component (that gets dumped into static files)
conceptually I would just like to say "please run this command based on these files, some build results will end up here"
Cool. We have a similar setup at Toolchain. Django app + JavaScript. We’re starting to look into adding JS support to V2 in the #v2node channel, but for now, you would want to use the V2 implementation for Python (those docs) and the V1 implementation for JavaScript. Are you already using Pants? If so, which version?
I'm not using it yet (this is still exploratory), I have v1.27.0rc0 installed
👍 1
But yeah, we already have a "normal" build system and Pants being Python is very tempting, but I would ideally just like to wrap the existing system in Pants (then split it up later) but not seeing an easy way to do this
Can you clarify what you mean by "wrap the existing system in Pants"?
Well, for example now I run
grunt build-js-files
. I would love to just specify a Pants ... target? That just calls that command (but I specify the files that are needed to run that)
Like we have 5 or 6 build steps already, I think we could use Pants and caching to get advantages from the "same input/same output" principle.
Likely, it would be easiest to create a simple plugin/new goal to do this. You could do something like
./pants grunt-build '**/*.js'
, for example You specifically need to run
grunt build-js-files
, right? Any other commands?
If I wrote out a new goal, would I then be able to write a
target so that
./pants test
reruns that build if I have input changes?
Well there's Django stuff like
manage.py compilemessages
(for translation files), I could write up wrapper files for this probably?
While we don’t yet have official V2 support for Node, I think we could easily create a plugin to run
grunt build-js-files
or something similar on some targets. An explicit focus of the V2 version of Pants is to make it highly extensible and easy to add new functionality. We’ll be adding docs on how to write plugins next month, and I’d be happy to pair on it too if you think the plugin would do the job.
Regarding Python, there is already support for what you’re describing. Pants will understand when the inputs to the test command haven’t changed and will leverage the cache for those tests. Another option is
./pants --changed-since=HEAD test
, which will find all the changed files sine that commit and only run those
When trying stuff out I was hitting things telling me that the dependencies to a
could only be
-like targets
Hm, I don’t think we have validation for that. What is the error message? Generally you would express that type of dependency via a
target. For example, you could do this:
Copy code
  sources=["*.css", "*.html", "*.js"]

So basically what happens here.... is: •
grunt build-js-files
and generates
manage.py compilemessages
generates some
file for translations in the app •
py.test app/tests
runs tests (including selenium tests, which requires the above built files to be present)
can I make a`files` declaration for stuff that is generated from some other command (like a plugin that would run the commands above)?
sorry for spamming on this, it's very nice to be able to get help on this kind of stuff though
declaration for stuff that is generated from some other command
Good timing with this question! We’re actually actively working on support for codegen the next two weeks, meaning for example that you can convert Thrift files into Python code and depend on that in your
targets, for example. We could possibly leverage that, but I think we might be able to circumvent it for something simpler. Generally, a V2 goal can indeed create files in your build root. You could create a
goal that will write files to any location in the build root, like
. Question, do you check in
to version control?
no, we don't commit it
👍 1
sorry for spamming on this, it’s very nice to be able to get help on this kind of stuff though
No need to apologize! Happy to help 🙂 our goal is to solve problems exactly like you’re encountering!
no, we don’t commit it
Okay, then codegen would likely be more idiomatic, but in the meantime, I think we could still make it work. The semi-unidiomatic thing is that you will want to check in your BUILD file to VCS, so you will want your BUILD file defined one level above
, like at the root level. Then, you could use a
glob, like this:
Copy code
  sources=["dist/main.js", "dist/assets/*.png"],
Your targets can then depend on
, where the
clarifies this target is at the build root.
Makes sense, I'll try to use this with a plugin or something to see if I can get things working. Definitely think that a general codegen entrypoint would make adoption of this tool much easier for existing projects 😃
💯 1
Are you thinking of trying to do a V1 plugin or V2? While we haven’t yet written the docs for V2 (it’s the next thing on my roadmap after I finish codegen), it would likely be much easier to implement and brings benefits like better caching. I’d be happy to pair with you on it 🙂 Beyond being happy to try to help, it would be useful feedback for me in writing the plugin docs on what makes sense and what is confusing.
I would be OK trying out the V2 plugin, is there some base Python class I can look at for that? And maybe an example file that's already in the project? I'm pretty good at back-solving from an existing implementation
What I'll do here is make a new repo with a simple version of my use case and I'll share the link in this Slack, and try to figure out the details
👍 1
And maybe an example file that’s already in the project? I’m pretty good at back-solving from an existing implementation
I’ll look for some good examples right now. Generally I’m the same with being good at back-solving, although warning that there are a fair amount of new concepts with writing V2 “rules” that might be confusing without a little bit of explanation. All the concepts are sound and inspired by functional programming, and once you know the basics it’s pretty simple to throw together new functionality, but it might be a little confusing without some orientation (whether that be the upcoming docs or someone talking it through). Of course, we’re happy to answer any questions along the way
is a good example of a stand-alone goal that does something similar to what you’ll want for the Grunt goal. Ignore lines 26-46. First step to your new goal: create a subclass of
, very similar to this file.
is the public api, like any options and the name of the goal. Then, the
is where the interesting logic happens. High level flow of what’s going on: 1) We’re asking for a couple things to be given to us by the V2 engine. These will be given via dependency injection; we don’t care how Pants gets us these things. 2) One of the things we’re asking for in the function signature is
. This is a way to ask for source files, where we don’t care at all about the metadata in targets. All we care about is the raw source files. If a user runs
./pants cloc :example
, we get all the source files belonging to the target
. If they say
./pants cloc example.js
, we’ll get that JS file. 3) We set up a
, which is very similar to Python’s
but is a way the Pants engine understands it and does things like safe caching. Note that we run
. For an initial prototype, it will be easiest if you can use an absolute path like
. We can help you to generify this once it works. Also, note the
to populate the chroot with the files we need, and
to “capture” the result. 4)
await Get[ProcessResult](Process, req)
actually runs the process -- The other relevant file is
, lines 69-71. It shows how to materialize files from a tmpdir into the actual build root so that you write to
plugin registration is the same as before yeah? I should be able to get stuff working through the plugin docs on the v1 documentation?
It’s similar in that you have a register.py file. The main entry point is rules() though. See backend/python/register.py as an example You also set backed_packages2 instead of backend_packages