is it a bug that experimental-deploy errors instead of doing nothing if there’s no targets to deploy?

Hi all, has anyone integrated a NodeJS project in Pants? Most of my packages are Python, but there’s a single NodeJS one I’d like to integrate too

Okey, this is starting to annoy me a great deal: Why does

./pants peek translations/de/LC_MESSAGES/messages.po

not go kaboom with the Please Run error? (see code in thread)

I have

class PortableObjectSourceField(SingleSourceField):
    expected_file_extensions = ('.po',)
    default = 'messages.po'


class PortableObjectDependenciesField(Dependencies):
    pass


class PortableObjectTarget(Target):
    alias = "portable_object_file"
    core_fields = (
        PortableObjectSourceField,
        PortableObjectDependenciesField,
        *COMMON_TARGET_FIELDS,
    )
    help = "A gettext Portable Object file."


@dataclass(frozen=True)
class PortableObjectFileInferenceFieldSet(FieldSet):
    required_fields = (PortableObjectSourceField, PortableObjectDependenciesField)

    source: PortableObjectSourceField
    dependencies: PortableObjectDependenciesField


class InferPortableObjectDependencies(InferDependenciesRequest):
    infer_from = PortableObjectFileInferenceFieldSet

@rule
async def infer_markup_files_for_portable_object_file(
    request: InferPortableObjectDependencies,
) -> InferredDependencies:
    raise RuntimeError("Please Run")

    return InferredDependencies([])

def rules() -> Iterable[Rule]:
    return [*collect_rules(), UnionRule(InferDependenciesRequest, InferPortableObjectDependencies)]

def target_types() -> Iterable[type[Target]]:
    return [PortableObjectTarget]

what I get is

[
  {
    "address": "translations/de/LC_MESSAGES:LC_MESSAGES",
    "target_type": "portable_object_file",
    "dependencies": [],
    "dependencies_raw": null,
    "description": null,
    "source_raw": "messages.po",
    "sources": [
      "translations/de/LC_MESSAGES/messages.po"
    ],
    "tags": null
  }
]

related https://pantsbuild.slack.com/archives/C01CQHVDMMW/p1674031011166129

Sorry for frustration but it's almost like the UnionRule is not being picked up and it's driving me nuts 🤪

well… I tried your code, and with this: git diff --staged diff --git a/src/python/pants/backend/demo/BUILD b/src/python/pants/backend/demo/BUILD new file mode 100644 index 000000000..0ec0a26db --- /dev/null +++ b/src/python/pants/backend/demo/BUILD @@ -0,0 +1,5 @@ +# Copyright 2023 Pants project contributors (see CONTRIBUTORS.md). +# Licensed under the Apache License, Version 2.0 (see LICENSE). +python_sources() + +portable_object_file(name=“testing”) diff --git a/src/python/pants/backend/demo/__init__.py b/src/python/pants/backend/demo/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/src/python/pants/backend/demo/messages.po b/src/python/pants/backend/demo/messages.po new file mode 100644 index 000000000..e69de29bb diff --git a/src/python/pants/backend/demo/register.py b/src/python/pants/backend/demo/register.py new file mode 100644 index 000000000..bb6119527 --- /dev/null +++ b/src/python/pants/backend/demo/register.py @@ -0,0 +1,64 @@ +# Copyright 2023 Pants project contributors (see CONTRIBUTORS.md). +# Licensed under the Apache License, Version 2.0 (see LICENSE). +from dataclasses import dataclass +from typing import Iterable + +from pants.engine.rules import Rule, UnionRule, collect_rules, rule +from pants.engine.target import ( + COMMON_TARGET_FIELDS, + Dependencies, + FieldSet, + InferDependenciesRequest, + InferredDependencies, + SingleSourceField, + Target, +) + + +class PortableObjectSourceField(SingleSourceField): + expected_file_extensions = (“.po”,) + default = “messages.po” + required = False + + +class PortableObjectDependenciesField(Dependencies): + pass + + +class PortableObjectTarget(Target): + alias = “portable_object_file” + core_fields = ( + PortableObjectSourceField, + PortableObjectDependenciesField, + *COMMON_TARGET_FIELDS, + ) + help = “A gettext Portable Object file.” + + +@dataclass(frozen=True) +class PortableObjectFileInferenceFieldSet(FieldSet): + required_fields = (PortableObjectSourceField, PortableObjectDependenciesField) + + source: PortableObjectSourceField + dependencies: PortableObjectDependenciesField + + +class InferPortableObjectDependencies(InferDependenciesRequest): + infer_from = PortableObjectFileInferenceFieldSet + + +@rule +async def infer_markup_files_for_portable_object_file( + request: InferPortableObjectDependencies, +) -> InferredDependencies: + raise RuntimeError(“Please Run”) + + return InferredDependencies([]) + + +def rules() -> Iterable[Rule]: + return [*collect_rules(), UnionRule(InferDependenciesRequest, InferPortableObjectDependencies)] + + +def target_types() -> Iterable[type[Target]]: + return [PortableObjectTarget]

I get:

./pants peek src/python/pants/backend/demo/messages.po
11:24:00.85 [WARN] DEPRECATED: `pants.engine.environment.Environment` is scheduled to be removed in version 2.17.0.dev0.

Use `pants.engine.env_vars.EnvironmentVars`.
11:24:00.85 [WARN] DEPRECATED: `pants.engine.environment.EnvironmentRequest` is scheduled to be removed in version 2.17.0.dev0.

Use `pants.engine.env_vars.EnvironmentVarsRequest`.
11:24:00.86 [WARN] DEPRECATED: `pants.engine.environment.CompleteEnvironment` is scheduled to be removed in version 2.17.0.dev0.

Use `pants.engine.env_vars.CompleteEnvironmentVars`.
11:24:00.91 [INFO] Initializing scheduler...
11:24:04.93 [INFO] Scheduler initialized.
11:24:05.62 [ERROR] 1 Exception encountered:

Engine traceback:
  in select
    ..
  in pants.backend.project_info.peek.peek
    `peek` goal

Traceback (most recent call last):
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 626, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/backend/project_info/peek.py", line 178, in peek
    tds = await Get(TargetDatas, UnexpandedTargets, targets)
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 120, in __await__
    result = yield self
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 626, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/backend/project_info/peek.py", line 141, in get_target_data
    dependencies_per_target = await MultiGet(
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 360, in MultiGet
    return await _MultiGet(tuple(__arg0))
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 167, in __await__
    result = yield self.gets
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 626, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/graph.py", line 1200, in resolve_dependencies
    inferred = await MultiGet(
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 360, in MultiGet
    return await _MultiGet(tuple(__arg0))
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 167, in __await__
    result = yield self.gets
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/engine/internals/selectors.py", line 626, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/andreas.stenius/src/github/kaos/pants/src/python/pants/backend/demo/register.py", line 54, in infer_markup_files_for_portable_object_file
    raise RuntimeError("Please Run")
RuntimeError: Please Run

only change to your code I did was add

required = False

to the source field in order to have it optional… seems setting

default = …

wasn’t enough. And the imports.

so, it would seem it has something to do with your setup, rather than you doing something wrong here…

Does it matter that you did not register it as a plugin?

Oh, not shown in the diff is:

git diff pants.toml
diff --git a/pants.toml b/pants.toml
index ed9942170..cecaac289 100644
--- a/pants.toml
+++ b/pants.toml
@@ -7,6 +7,7 @@ backend_packages.add = [
   "pants.backend.build_files.fix.deprecations",
   "pants.backend.build_files.fmt.black",
   "pants.backend.python",
+  "pants.backend.demo",
   "pants.backend.experimental.python.packaging.pyoxidizer",
   "pants.backend.python.lint.autoflake",
   "pants.backend.python.lint.black",

right, what I mean is the fact that it is "in repo" has no bearing?

no, that should not matter

I can get other rules to run for the same target and others, we have a fair share of plugins, just not

UnionRule(InferDependenciesRequest, ...)

will not be executed

upgrading to pants 2.14 fixes the problem, so there is that 🙂

so, if upgrade is undesirable at this point, maybe try the

InjectDependenciesX

if those are around in your version of Pants.. ?

I'll take the excuse to upgrade!

We're on 2.13.1 so can't be that bad a migration

I’m surprised that this type existed but didn’t work on that version… sounds like a bug to me.

Would the type being present in source but not working also explain why tests where I explicitly constructed the QueryRule made it "just work"?

Hey Pants, looking to integrate a project in an existing monorepo containing a single monolith. I'd like to keep these project separate at the dependency level too. Is it possible to define something like "for this source root, always use these thirdparty dependencies"?. I'm currently facing errors like this

pants.backend.python.dependency_inference.rules.UnownedDependencyError: Pants cannot infer owners for the following imports in the target app/monolith/something/models.py:

  * psycopg2 (line: 8)

This makes sense because I added a new pyproject.toml for the new project but curious if there a way around this without impacting the monolith dependencies.

possibly. I’ve not looked at why it doesn’t work without it…

Ok, so I think I have experienced a pants bug, but am having trouble reproducing and narrowing down exactly what's happening. Very occasionally, when running

./pants check

, I will get a failure on a file that shouldn't fail. This seems to always happen on files that are part of more than one resolve and the failure has something to do with missing type stubs. Here are the problems I have had with trying to figure this out: 1. I have tried to reproduce it by running with various forms of caching on and off but cannot find a way to reliably get this to happen or narrow the problem down. 2. the file I get an error on complains about missing type stubs for a library it does not depend on, even transitively Has anyone else seen this? Any suggestions on how to dig into this?

The stricter you get in terms of python package version i.e. pinning exact version vs using something like

~major.minor

, longer it takes to generate lockfiles. What’s the recommendation here?

Hello everyone. Quick question about the linters report folder. We are using pants version 2.16.0.dev4, and for bandit and pylint (not flake8), there are 2 generated folders in

dist/lint/<linter_name>

:

19:00:25.65 [INFO] Wrote lint report files to dist/lint/bandit/__CPython==3.9.13__.
19:00:25.65 [INFO] Wrote lint report files to dist/lint/bandit/__CPython==3.9.13___.
19:00:25.65 [INFO] Wrote lint report files to dist/lint/flake8.
19:00:25.65 [INFO] Wrote lint report files to dist/lint/pylint/__CPython==3.9.13__.
19:00:25.65 [INFO] Wrote lint report files to dist/lint/pylint/__CPython==3.9.13___.

I don't remember having this behavior with 2.14.x. Do you have any idea what can cause this?

Whaaa?

So some reports are going to

__CPython==3.9.13__

and some reports to that but with an extra

_

on the end?

That does not seem intentional...

Good thing this should be easy to grep. Someone look for an underscore in the Pants codebase.... 😢