I have a small question about custom plugins, I tr...
# general
f
I have a small question about custom plugins, I tried to search Slack, docs and example repos I hope someone can pointout what I am doing wrong. I want to able to get a package's version built in CI with a tag. so if the package's name is
test_package
I want to be able to print the right tag(will be same as the git tag of that release) with
print(test_package.version)
After researching a little bit I found it can be done with a custom plugin (similar to a codegen plugin) by creating a
version.py
file with the contents of a env var
BUILD_TAG
defined when packaging. where the
__init__.py
of the package will be like this:
Copy code
from .version import __version__
version = __version__

__all__ = ['version', "__version__"]
I am able to build this plugin, it works fine but throws a warning like this:
Copy code
16:33:44.06 [WARN] Unmatched glob from test_package:version's `source` field: "test_package/version.py"

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.18/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
This is the plugin
register.py
in nutshell
Copy code
"""Version tag plugin."""

import logging
import os
from typing import Any, cast

from pants.backend.python.target_types import PythonSourceField
from pants.engine.fs import (
    AddPrefix,
    CreateDigest,
    Digest,
    FileContent,
    Snapshot,
)
from pants.engine.rules import Get, collect_rules, rule
from pants.engine.target import (
    COMMON_TARGET_FIELDS,
    Dependencies,
    GeneratedSources,
    GenerateSourcesRequest,
    OptionalSingleSourceField,
    StringField,
    Target,
)
from pants.engine.unions import UnionRule
from pants.source.source_root import SourceRoot, SourceRootRequest
from pants.util.logging import LogLevel

logger = logging.getLogger(__name__)


class VersionFileSourceField(OptionalSingleSourceField):
    """Version file to write to."""

    default = "version.py"
    help = "File to store version (eg. `version.py`)."
    # required = True


class VersionField(StringField):
    """Version to write to file."""

    alias = "version_tag"
    help = "Version to write to file."
    required = True


class VersionFileSourceTarget(Target):
    """A single version file target."""

    alias = "write_version_file"
    help = "A single version file."
    core_fields = (*COMMON_TARGET_FIELDS, Dependencies, VersionFileSourceField)


class GeneratePythonVersionFileRequest(GenerateSourcesRequest):
    """Generate a Python version file from a version file target."""

    input = VersionFileSourceField
    output = PythonSourceField


@rule(level=LogLevel.DEBUG)
async def write_version_file_func(
    request: GeneratePythonVersionFileRequest,
) -> GeneratedSources:
    """Write a version file from a version file target.

    Parameters
    ----------
    request
        The request.
    """
    package_name = request.protocol_target.address.spec_path
    output_filename = cast(str, request.protocol_target[VersionFileSourceField].value)
    version_tag = request.protocol_target[VersionField].value

    logger.debug(f"package_name: {package_name}")
    logger.debug(f"output_filename: {output_filename}")
    logger.debug(f"version_tag: {version_tag}")

    digest = await Get(
        Digest,
        CreateDigest(
            [
                FileContent(
                    os.path.join(package_name, output_filename),
                    f"version = '{version_tag}'\n".encode(),
                )
            ]
        ),
    )

    source_root = await Get(
        SourceRoot,
        SourceRootRequest,
        SourceRootRequest.for_target(request.protocol_target),
    )

    source_root_restored = (
        await Get(Snapshot, AddPrefix(digest, source_root.path))
        if source_root.path != "."
        else await Get(Snapshot, Digest, digest)
    )
    return GeneratedSources(source_root_restored)


def target_types() -> list[type[Target]]:
    """Return the list of target types this plugin registers."""
    return [VersionFileSourceTarget]


def rules() -> Any:
    """Return the list of rules this plugin registers."""
    return [
        *collect_rules(),
        UnionRule(GenerateSourcesRequest, GeneratePythonVersionFileRequest),
        VersionFileSourceTarget.register_plugin_field(VersionField),
    ]
and the
BUILD
for any package in this mono repo will use it like this:
Copy code
write_version_file(
    name="version",
    source="version.py",
    version_tag=env("BUILD_TAG", "undefined_build_tag_env"),
)

python_sources(
    name="test_package",
    dependencies=[":version"],
)
h
You've defined your version.py as a source file, but it isn't one - it's generated at build time.
So Pants is rightfully complaining about it not existing on disk, as is expected for sources.
In codegen the thing you generate from (e.g., a .proto file) is a source, but the thing you generate from it is not
So basically you don't want to use OptionalSingleSourceField for this
f
I see. but if I dont use the SourceField the plugin is never triggered. Is there a way to trigger this plugin otherwise ?
I think then I cant use the
GenerateSourcesRequest
right?
h
I don't remember the details, but take a look at the implementation of the
vcs_version
target in the python backend.
In fact, it occurs to me that you might be able to just use that target without a plugin
f
hm., I tried using it, but didnt work for some reason (I think it doesnt write to a file), but yea, I didn't look at the source of that. thanks for helping me with this. hope you have a great day 🙂
h
So whether you can use it directly or need to roll your own, it is an example of "codegen" that doesn't originate with a source file but with git state
so it's probably a useful thing to copy from
👍 1
f
This looks interesting for sure, vcs_version uses
VCSVersionDummySourceField
, I think this will work for my usecase. I'll keep you posted on how this goes..
VCSVersionDummySourceField
is a subclass of
OptionalSourceField
so vcs_version is doing the same thing as I did. But after playing with it a little bit, referencing the vcs_version I found out that I shouldn't use the
source
field of the optionalSourceField, so it acts like a dummy field. and use
alias
with an
_
prefix to hide that field from help, this works as expected for my usecase.
Thank you for the help @happy-kitchen-89482!
h
Great
Glad to hear
🙌 1