Travis doesn't appear to have triggered for my release (or at all in the last ~48 hours)? https://travis-ci.com/github/pantsbuild/pants/builds

Does Pants itself, or a script within the pants repo, provide a utility for removing unused Python imports?

It's very annoying that flake8 will happily complain about them but not just remove them for me in the

fmt

goal

I'm trying to build Pants locally at 1.30.x and getting this error:

warning: src/zlib/gzlib.c:214:15: error: implicitly declaring library function 'snprintf' with type 'int (char *, unsigned long, const char *, ...)' [-Werror,-Wimplicit-function-declaration]
warning:         (void)snprintf(state->path, len + 1, "%s", (const char *)path);
warning:               ^
warning: src/zlib/gzlib.c:214:15: note: include the header <stdio.h> or explicitly provide a declaration for 'snprintf'
warning: 1 error generated.

error: failed to run custom build command for `libz-sys v1.0.25`

Ring any bells?

Hm, I think I've managed to anger the engine again. It seems like if I add

UnionRule(TestFieldSet, JavaTestFieldSet)

, I somehow indirectly break Python's test rule graph.

No installed rules return the type PythonTestFieldSet, and it was not provided by potential callers of @rule(pants.backend.python.goals.pytest_runner:282:run_python_test(PythonTestFieldSet, TestSubsystem, PyTest) -> TestResult, gets=[Get(TestSetup, TestSetupRequest), Get(FallibleProcessResult, Process), Get(PySnapshot, DigestSubset), Get(PySnapshot, AddPrefix), Get(PySnapshot, RemovePrefix)]).

Where

JavaTestFieldSet

is a very minimal crib off of

PythonTestFieldSet

:

@dataclass(frozen=True)
class JavaTestFieldSet(TestFieldSet):
    required_fields = (JavaTestsSources,)

    sources: JavaTestsSources

I'm not surprised that my testing code doesn't work, or maybe isn't using the engine properly, but I am surprised that the symptom appears to be a complaint about Python rules, which I'm not even importing

I don't expect it to otherwise be working, but in case it helps, here is my current working state: https://github.com/pantsbuild/pants/commit/52906dbc778da1c175422f2d1906d08d52667a8d#diff-ca5c5012c3bde561e871385ac[…]3f681b44a5dcd5afd5299092f8cf34f

On the 1.30.x branch some of

test_export.py

fail, both in CI and locally: https://travis-ci.com/github/pantsbuild/pants/jobs/496026410

Anyone enjoy regex and interested in helping with this? I want to improve our dep inference for Protobuf imports so that it handles

//

comments:

Started work on adding

output-format

to

dependencies

goal. Seeking early redirection with a WIP PR.

--ignore-pants-warnings

expects regex patterns, which can result in surprising behavior like

The option

[isort].config

is not configured

not actually ignoring that warning due to

[]

not being escaped. Are folks using the regex feature? I'd like to simplify with a new option

--ignore-warnings

that is normal strings, and deprecate the old one.

So now I'm into a new failure mode, where again I thought I'd cribbed the pytest structure basically exactly:

File "/home/patrick/code/pants/src/python/pants/engine/target.py", line 839, in create
    address=tgt.address, **_get_field_set_fields_from_target(cls, tgt)
TypeError: __init__() missing 1 required positional argument: 'sources'

That's with

@dataclass(frozen=True)
class JavaTestFieldSet(TestFieldSet):
    required_fields = (JavaTestsSources,)

    sources: JavaTestsSources

And the target looks like this:

class JavaSources(Sources):
    expected_file_extensions = (".java",)


class JavaTestsSources(JavaSources):
    default = ("*Test.java",)


class JavaTests(Target):
    alias = "java_tests"
    core_fields = (
        *COMMON_TARGET_FIELDS,
        Dependencies,
        JavaTestsSources,
    )
    help = "Java tests"

CI started flaking with this error since landing Shellcheck:

Exception: String("Cannot strip prefix __output from root directory (Digest with hash Fingerprint<bb11eda082823e90c95f959f8b89c6886423521773bb24bb31e8c25454d0740b>) - root directory didn\'t contain a directory named __output but did contain directory named: shellcheck-v0.7.1")

https://github.com/pantsbuild/pants/runs/2291026549#step:9:234 The confusing thing to me is why it's flaky

is this a known flake ? https://github.com/pantsbuild/pants/runs/2300092374?check_suite_focus=true#step:8:1129

@enough-analyst-54434 thoughts on us changing the default for

pex_binary

to

venv

, rather than

zipapp

? Fwict, it has far fewer gotchas and much better perf for most use cases. It will require a deprecation cycle to change the default. I can do that if you agree

cont. from https://pantsbuild.slack.com/archives/CASMF8SJ1/p1617029155053700 regarding https://github.com/pantsbuild/pants/issues/11808 @hundreds-father-404 quick question, if while doing dependency inference, shouldn’t it be an error if we can’t resolve a dependency? Looking at this line in a test case: https://github.com/pantsbuild/pants/blob/bb3fa05d4b32512eb8674d6dbfacb5f07e1f8596/src/python/pants/backend/python/goals/setup_py_test.py#L159

pex_binary(name="bin", entry_point="foo.qux.bin:main")

I don’t see that there is a

bin.py

file in

foo.qux

for the test, and it is not listed in the assert either, so I feel like this should fail.. but seems like it is silently ignored?

ready for some review feedback: https://github.com/pantsbuild/pants/pull/11872 😉 (no rush, I’m going afk now..)

seeing this one a clean master checkout:

$ ./pants goals
Building native engine
error: invalid channel name '[toolchain]' in '/Users/yic/workspace/pants_2/rust-toolchain'
info: caused by: invalid toolchain name: '[toolchain]'

Failed to build native engine.

any advice?

Do we have a convention for documenting attributes in dataclasses? e.g. using a docstring after the attribute line, or just having an

__init__

-style docstring at the class level?

So I'm trying to figure out if I should using

append_only_caches

in my Coursier code, but I'm still not 100% sure what they're supposed to be used for

I see in the Rust documentation:

/// These caches are globally shared and so must be concurrency safe: a consumer of the cache

/// must never assume that it has exclusive access to the provided directory.

What does "global" actually mean here? I'm assuming it's global to a given pants repo, or maybe pantsd executor, but maybe it means global to a machine? My best guess is that in a scenario with remote execution, the remote worker would be accumulating and persisting this cache across invocations

If that's the case, I'm not sure if I should bother plumbing it to Coursier. Letting Coursier use its default global cache (

~/.cache/coursier

) seems equally safe but with hypothetically better cache hits--unless I'm missing something here

OK, now I'm into writing tests for rules! I'm using an existing test as a reference and trying to base mine off of it: https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/lint/black/rules_integration_test.py My issue is that I'm trying to test a non-Root rule, or I guess a non-

QueryRule

. At least, it doesn't have a directly associated

QueryRule

the way I wrote it, and I don't think it should have one. It's an intermediate rule and product used by others. Do I just need to stub in a

QueryRule

to force the

RuleRunner

to have a path from the root query to my rule?

Another testing question: is there an easy way to mock the behavior of a

Process

invocation that I know my rule is going to make?

Is there a technical reason for native engine types (e.g.

PyDigest

) not having a

___repr___

on the Python side of things?

I guess part of the contract for getting a

Digest

is that the engine is guaranteeing that there's actually a blob to back that up in the DB? Whereas if you can just construct it by hand from a fingerprint and length and pass it around, you might violate some invariant in the engine