https://pantsbuild.org/ logo
Join Slack
Powered by
Hi there! I'm just converting the existing codebas...
# general
n
Hi there! I'm just converting the existing codebase into a monorepo based on Pants. Before, each Python package lived in its own repository and used git tags to get its version. I just discovered that version only in the BUILD file is not enough - some of the code needs access to the version even when not installed so it would be ideal if I can put a version into a VERSION file so any code (including generated setup.py) can read it from there. How can one get custom code into a generated setup.py? Do I need to write a custom target type for this?
h
Hi! Welcome to Pants 🙂 Indeed, great timing with asking for something like this around the same time as #10638 being opened. Are you okay with manually updating those VERSION files? If so, then the linked comment will work (with one tweak I can send). Otherwise, if you want something automatic like Adrian is working on, it’d need to be use the more powerful Rules API.
h
To follow up, we'll have a solution for this out in the next couple of days, but can you clarify how you would like to create the version strings for your packages?
Is the idea that each package has its own VERSION file? What would populate that file?
n
@hundreds-father-404 I'm definitely ok with manual updates of VERSION files. Please send the tweak, I will give it a try 🙂 Many thanks!
@happy-kitchen-89482 Yes, the idea is that each package will has its own VERSION file and those will be updated manually. This is because packages will be released separately.
h
OK, so that should be easy to do with existing macros (but you will need to add a dependency from the 
python_distribution
target to a 
resource
target that owns the 
VERSION
file, so that invalidation happens when the VERSION file changes.
h
Disregard that part about the old-style macros. We’re nearly done with a new mechanism for doing what you’re asking. It handles caching more robustly, rather than relying on hacks; and it uses the newer Target and Rules API like all other plugins do. We’re going to use that mechanism for our Pants distributions, which also read from a VERSION file, so the example will be very similar to yours. https://github.com/pantsbuild/pants/pull/10721 adds the mechanism and https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal is WIP docs about how to use it. I hope to land this tomorrow, and get it in the alpha 2 release on Friday
Hey Zdeněk, we released 2.0.0a2 last night, which has this new hook for 
setup-py
kwargs. It includes an example that does exactly what you’re trying to do. Start with this guide to upgrade to 2.0: https://www.pantsbuild.org/v2.0/docs/how-to-upgrade-pants-2-0 Then read https://www.pantsbuild.org/v2.0/docs/plugins-overview for an overview of how the Plugin API works, including its link to the concepts for the Rules API. Then, you can follow https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal for a guide on 
setup-py
. Let us know how we can help with this! Those docs are fairly new and not very battle tested, so any feedback is appreciated
n
Great, it works!
❤️ 1
In https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal#2-create-a-rule-with-your-logic, I just found out that
Copy code
@rule
def setup_kwargs_plugin(request: CustomSetupKwargsRequest) -> SetupKwargs:
...should be
Copy code
@rule
async def setup_kwargs_plugin(request: CustomSetupKwargsRequest) -> SetupKwargs:
Otherwise docs seems to be ok.
h
Ohh, good catch. Thanks! Any feedback on the Rules API in general, such as the Rules API concepts page? I’m wondering how much those concepts made sense vs flying over your head. Very few people have used the Rules API beyond the main contributors because it was so experimental before 2.0. We now know the ins and outs of the API, so I’m curious what it’s like to use the first time and what parts are confusing.
n
Sorry, forget to reply... Well, to be honest, I just rushed through the documentation and hacked the solution together somehow. But what I was missing most was links to API references - I wasn't sure what are available methods/properties of request, target, address...
Btw, is there a way how can I get name of setup_py target? I mean how can I get package name / "arcor2_execution" in the following case?
Copy code
python_distribution(
    name="arcor2_execution_dist",
    dependencies=[
        ":arcor2_execution",
        ":py.typed"
    ],
    provides=arcor2_setup_py(
        name="arcor2_execution",
        description="ARCOR2 - Augmented Reality Collaborative Robot."
    ).with_binaries(
        {
            "arcor2_execution": "src/python/arcor2_execution/scripts:execution"
            }
        )
)
As a temporary solution I used
Copy code
package_name = remove_suffix(request.target.address.target_name, "_dist")
...but there has to be better way 🙂
And how can I do some logging from plugin?
h
Btw, is there a way how can I get name of setup_py target?
Yes, sorry, this is one of the remaining parts I need to document. So, 
request.target
is a 
Target
, which stores a bunch of fields like the 
provides
field. You want to access the 
provides
field. To do that, write 
request.target[PythonProvidesField]
, where 
PythonProvidesField
is imported from 
pants.backend.python.target_types
. This will give you back a 
Field
(capitalized), which has the properties 
.alias
and 
.value
. You want 
.value
, to get the actual value the user gave. The 
value
for a 
PythonProvidesField
is a little more complex than normal. Typically, `Field`s store primitives like 
str
and 
int
. For this one, it stores a 
PythonArtifact
object, which has a property 
.name
. So, in full, you’ll do 
request.target[PythonProvidesField].value.name
👍 1
I was missing most was links to API references
Agreed that would be nice. We have hand-written references for some things like filesystem operations, but indeed nothing generated. For now, the alternative is to go to the definition in source code, e.g. if your editor lets you jump to the definition. We try to write good docstring, and we zealously use type hints
Okay I updated the docs on Rules and the Target API. PTAL 🙂 https://www.pantsbuild.org/v2.0/docs/rules-api-and-target-api I also started a new page about logging and rules. The part relevant to your question is complete. I’ll work on the rest this next week. https://www.pantsbuild.org/v2.0/docs/rules-api-logging
n
Thank you for the hints, I finally got some time to improve my plugin a bit.
3 Views
Open in Slack
PreviousNext
Powered by