https://pantsbuild.org/ logo
#general
Title
# general
n

narrow-activity-17405

08/31/2020, 8:02 AM
Hi there! I'm just converting the existing codebase into a monorepo based on Pants. Before, each Python package lived in its own repository and used git tags to get its version. I just discovered that version only in the BUILD file is not enough - some of the code needs access to the version even when not installed so it would be ideal if I can put a version into a VERSION file so any code (including generated setup.py) can read it from there. How can one get custom code into a generated setup.py? Do I need to write a custom target type for this?
h

hundreds-father-404

08/31/2020, 5:10 PM
Hi! Welcome to Pants 🙂 Indeed, great timing with asking for something like this around the same time as #10638 being opened. Are you okay with manually updating those VERSION files? If so, then the linked comment will work (with one tweak I can send). Otherwise, if you want something automatic like Adrian is working on, it’d need to be use the more powerful Rules API.
h

happy-kitchen-89482

09/01/2020, 4:25 AM
To follow up, we'll have a solution for this out in the next couple of days, but can you clarify how you would like to create the version strings for your packages?
Is the idea that each package has its own VERSION file? What would populate that file?
n

narrow-activity-17405

09/01/2020, 7:29 AM
@hundreds-father-404 I'm definitely ok with manual updates of VERSION files. Please send the tweak, I will give it a try 🙂 Many thanks!
@happy-kitchen-89482 Yes, the idea is that each package will has its own VERSION file and those will be updated manually. This is because packages will be released separately.
h

happy-kitchen-89482

09/01/2020, 9:32 PM
OK, so that should be easy to do with existing macros (but you will need to add a dependency from the
python_distribution
target to a
resource
target that owns the
VERSION
file, so that invalidation happens when the VERSION file changes.
h

hundreds-father-404

09/02/2020, 7:16 AM
Disregard that part about the old-style macros. We’re nearly done with a new mechanism for doing what you’re asking. It handles caching more robustly, rather than relying on hacks; and it uses the newer Target and Rules API like all other plugins do. We’re going to use that mechanism for our Pants distributions, which also read from a VERSION file, so the example will be very similar to yours. https://github.com/pantsbuild/pants/pull/10721 adds the mechanism and https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal is WIP docs about how to use it. I hope to land this tomorrow, and get it in the alpha 2 release on Friday
Hey Zdeněk, we released 2.0.0a2 last night, which has this new hook for
setup-py
kwargs. It includes an example that does exactly what you’re trying to do. Start with this guide to upgrade to 2.0: https://www.pantsbuild.org/v2.0/docs/how-to-upgrade-pants-2-0 Then read https://www.pantsbuild.org/v2.0/docs/plugins-overview for an overview of how the Plugin API works, including its link to the concepts for the Rules API. Then, you can follow https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal for a guide on
setup-py
. Let us know how we can help with this! Those docs are fairly new and not very battle tested, so any feedback is appreciated
n

narrow-activity-17405

09/07/2020, 2:06 PM
Great, it works!
❤️ 1
In https://www.pantsbuild.org/v2.0/docs/plugins-setup-py-goal#2-create-a-rule-with-your-logic, I just found out that
Copy code
@rule
def setup_kwargs_plugin(request: CustomSetupKwargsRequest) -> SetupKwargs:
...should be
Copy code
@rule
async def setup_kwargs_plugin(request: CustomSetupKwargsRequest) -> SetupKwargs:
Otherwise docs seems to be ok.
h

hundreds-father-404

09/07/2020, 2:44 PM
Ohh, good catch. Thanks! Any feedback on the Rules API in general, such as the Rules API concepts page? I’m wondering how much those concepts made sense vs flying over your head. Very few people have used the Rules API beyond the main contributors because it was so experimental before 2.0. We now know the ins and outs of the API, so I’m curious what it’s like to use the first time and what parts are confusing.
n

narrow-activity-17405

09/11/2020, 10:02 AM
Sorry, forget to reply... Well, to be honest, I just rushed through the documentation and hacked the solution together somehow. But what I was missing most was links to API references - I wasn't sure what are available methods/properties of request, target, address...
Btw, is there a way how can I get name of setup_py target? I mean how can I get package name / "arcor2_execution" in the following case?
Copy code
python_distribution(
    name="arcor2_execution_dist",
    dependencies=[
        ":arcor2_execution",
        ":py.typed"
    ],
    provides=arcor2_setup_py(
        name="arcor2_execution",
        description="ARCOR2 - Augmented Reality Collaborative Robot."
    ).with_binaries(
        {
            "arcor2_execution": "src/python/arcor2_execution/scripts:execution"
            }
        )
)
As a temporary solution I used
Copy code
package_name = remove_suffix(request.target.address.target_name, "_dist")
...but there has to be better way 🙂
And how can I do some logging from plugin?
h

hundreds-father-404

09/11/2020, 5:06 PM
Btw, is there a way how can I get name of setup_py target?
Yes, sorry, this is one of the remaining parts I need to document. So,
request.target
is a
Target
, which stores a bunch of fields like the
provides
field. You want to access the
provides
field. To do that, write
request.target[PythonProvidesField]
, where
PythonProvidesField
is imported from
pants.backend.python.target_types
. This will give you back a
Field
(capitalized), which has the properties
.alias
and
.value
. You want
.value
, to get the actual value the user gave. The
value
for a
PythonProvidesField
is a little more complex than normal. Typically, `Field`s store primitives like
str
and
int
. For this one, it stores a
PythonArtifact
object, which has a property
.name
. So, in full, you’ll do
request.target[PythonProvidesField].value.name
👍 1
I was missing most was links to API references
Agreed that would be nice. We have hand-written references for some things like filesystem operations, but indeed nothing generated. For now, the alternative is to go to the definition in source code, e.g. if your editor lets you jump to the definition. We try to write good docstring, and we zealously use type hints
Okay I updated the docs on Rules and the Target API. PTAL 🙂 https://www.pantsbuild.org/v2.0/docs/rules-api-and-target-api I also started a new page about logging and rules. The part relevant to your question is complete. I’ll work on the rest this next week. https://www.pantsbuild.org/v2.0/docs/rules-api-logging
n

narrow-activity-17405

11/04/2020, 8:43 AM
Thank you for the hints, I finally got some time to improve my plugin a bit.
3 Views