Hi there! I'm just converting the existing codebas...
# general
Hi there! I'm just converting the existing codebase into a monorepo based on Pants. Before, each Python package lived in its own repository and used git tags to get its version. I just discovered that version only in the BUILD file is not enough - some of the code needs access to the version even when not installed so it would be ideal if I can put a version into a VERSION file so any code (including generated setup.py) can read it from there. How can one get custom code into a generated setup.py? Do I need to write a custom target type for this?
Hi! Welcome to Pants 🙂 Indeed, great timing with asking for something like this around the same time as #10638 being opened. Are you okay with manually updating those VERSION files? If so, then the linked comment will work (with one tweak I can send). Otherwise, if you want something automatic like Adrian is working on, it’d need to be use the more powerful Rules API.
To follow up, we'll have a solution for this out in the next couple of days, but can you clarify how you would like to create the version strings for your packages?
Is the idea that each package has its own VERSION file? What would populate that file?
@hundreds-father-404 I'm definitely ok with manual updates of VERSION files. Please send the tweak, I will give it a try 🙂 Many thanks!
@happy-kitchen-89482 Yes, the idea is that each package will has its own VERSION file and those will be updated manually. This is because packages will be released separately.
OK, so that should be easy to do with existing macros (but you will need to add a dependency from the
target to a
target that owns the
file, so that invalidation happens when the VERSION file changes.
Disregard that part about the old-style macros. We’re nearly done with a new mechanism for doing what you’re asking. It handles caching more robustly, rather than relying on hacks; and it uses the newer Target and Rules API like all other plugins do. We’re going to use that mechanism for our Pants distributions, which also read from a VERSION file, so the example will be very similar to yours. https://github.com/pantsbuild/pants/pull/10721 adds the mechanism and https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal is WIP docs about how to use it. I hope to land this tomorrow, and get it in the alpha 2 release on Friday
Hey Zdeněk, we released 2.0.0a2 last night, which has this new hook for
kwargs. It includes an example that does exactly what you’re trying to do. Start with this guide to upgrade to 2.0: https://www.pantsbuild.org/v2.0/docs/how-to-upgrade-pants-2-0 Then read https://www.pantsbuild.org/v2.0/docs/plugins-overview for an overview of how the Plugin API works, including its link to the concepts for the Rules API. Then, you can follow https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal for a guide on
. Let us know how we can help with this! Those docs are fairly new and not very battle tested, so any feedback is appreciated
Great, it works!
❤️ 1
In https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal#2-create-a-rule-with-your-logic, I just found out that
Copy code
def setup_kwargs_plugin(request: CustomSetupKwargsRequest) -> SetupKwargs:
...should be
Copy code
async def setup_kwargs_plugin(request: CustomSetupKwargsRequest) -> SetupKwargs:
Otherwise docs seems to be ok.
Ohh, good catch. Thanks! Any feedback on the Rules API in general, such as the Rules API concepts page? I’m wondering how much those concepts made sense vs flying over your head. Very few people have used the Rules API beyond the main contributors because it was so experimental before 2.0. We now know the ins and outs of the API, so I’m curious what it’s like to use the first time and what parts are confusing.
Sorry, forget to reply... Well, to be honest, I just rushed through the documentation and hacked the solution together somehow. But what I was missing most was links to API references - I wasn't sure what are available methods/properties of request, target, address...
Btw, is there a way how can I get name of setup_py target? I mean how can I get package name / "arcor2_execution" in the following case?
Copy code
        description="ARCOR2 - Augmented Reality Collaborative Robot."
            "arcor2_execution": "src/python/arcor2_execution/scripts:execution"
As a temporary solution I used
Copy code
package_name = remove_suffix(request.target.address.target_name, "_dist")
...but there has to be better way 🙂
And how can I do some logging from plugin?
Btw, is there a way how can I get name of setup_py target?
Yes, sorry, this is one of the remaining parts I need to document. So,
is a
, which stores a bunch of fields like the
field. You want to access the
field. To do that, write
, where
is imported from
. This will give you back a
(capitalized), which has the properties
. You want
, to get the actual value the user gave. The
for a
is a little more complex than normal. Typically, `Field`s store primitives like
. For this one, it stores a
object, which has a property
. So, in full, you’ll do
👍 1
I was missing most was links to API references
Agreed that would be nice. We have hand-written references for some things like filesystem operations, but indeed nothing generated. For now, the alternative is to go to the definition in source code, e.g. if your editor lets you jump to the definition. We try to write good docstring, and we zealously use type hints
Okay I updated the docs on Rules and the Target API. PTAL 🙂 https://www.pantsbuild.org/v2.0/docs/rules-api-and-target-api I also started a new page about logging and rules. The part relevant to your question is complete. I’ll work on the rest this next week. https://www.pantsbuild.org/v2.0/docs/rules-api-logging
Thank you for the hints, I finally got some time to improve my plugin a bit.