pantsbuild #development

Channels

welcome

github-notifications

development

wide-midnight-78598

04/06/2023, 7:36 PM

Well, this is frustrating 🤦‍♂️

Copy code

The `@rule_helper` decorator is no longer needed. `@rule` methods may call any other methods, and if they are `async` may also use `Get` and `MultiGet`.

Copy code

native_engine.IntrinsicError: Get(TargetRootsToFieldSets, TargetRootsToFieldSetsRequest, TargetRootsToFieldSetsRequest(field_set_superclass=<class 'pants.core.goals.package.PackageFieldSet'>, goal_description='', no_applicable_targets_behavior=<NoApplicableTargetsBehavior.ignore: 'ignore'>, shard=0, num_shards=-1)) was not detected in your @rule body at rule compile time. Was the `Get` constructor called in a separate function, or perhaps dynamically? If so, it must be inlined into the @rule body.

✅ 1

🤦 1

ancient-vegetable-10556

04/06/2023, 7:52 PM

I’m regularly getting test timeouts on

streaming_workunit_handler_integration_test

— known issue, or is this just on my branch? https://github.com/pantsbuild/pants/actions/runs/4631159201/jobs/8195999699

wide-midnight-78598

04/06/2023, 8:57 PM

Another fun rule graph error - that shows up on the Pants main repo, when trying to run anything - but doesn't show up when I use

pants_from_sources

on my plugins repo.

wide-midnight-78598

04/07/2023, 2:54 PM

Who would know how the pants docs search engine is loaded? I feel like https://www.pantsbuild.org/docs/reference-global isn't being indexed correctly. I've never really had success searching items on there

proud-dentist-22844

04/07/2023, 7:30 PM

I assume the answer is legacy: Why is

src/python/pants/backends/python/goals/setup_py.py

called

setup_py.py

instead of something like

package_dists.py

✅ 1

average-vr-56795

04/09/2023, 6:26 PM

https://nodejs.org/api/single-executable-applications.html Self-contained node apps, looks interesting!

average-vr-56795

04/10/2023, 7:19 AM

Aah apologies, I totally missed a dev release on Friday among Easter holiday times @ancient-vegetable-10556 What's the desired behaviour from the

EnvironmentBehavior

deprecation? Defaulting to

USES_ENVIRONMENTS

? Removing a default and requiring the property to be overridden in every

Goal

subclass?

bitter-ability-32190

04/10/2023, 11:42 AM

I've been musing that we could probably be more careful with core/engine changes while still delivering plugin features and improvements if we published the backends as packages. Is that something y'all think would be doable? Should I write something up?

bitter-ability-32190

04/10/2023, 3:35 PM

Update: • I was out, so last thing I remember was Stu and I got the "hardlink" change in. We need to tune the metric. It's likely we can lower it drastically without much overhead (meaning better perf). • Plans (most are blocking 2.16): ◦

ruff lint

support ◦ Benchmark

ruff

on one-file-per-process. Either way this thing is fast. But maybe this is even better? I think it'll depend on the speed of the scheduler honestly ◦ Add

pyenv

env var support ◦ Fix a

pyenv

IC selection bug. Tool ICs might be different than user ICs. We should try and pick one and only one if possible ◦ Deprecate/Remove

SecondaryOwnerMixin

. This is hazardous ◦ Add

CODEOWNERS

to the repo

bitter-ability-32190

04/10/2023, 6:12 PM

And while I'm on the idea train. I was starting to think of adding union "hooks" in common places (pre/post goals, etc...). My reasoning was I want to upstream a plugin, but it's usage (primarily the args) is a bit special to our org. So I want to have a hook telling people how to proceed on failure. Another usage could be the

lint

failure message(s): `(One or more formatters failed. Run

pants fmt

to fix.)` Anywho, food for thought 🙂

wide-midnight-78598

04/10/2023, 7:00 PM

@curved-television-6568 While this may become moot in a couple weeks/months, was there a way to extract the BUILD file where a target is defined, via peek?

bitter-ability-32190

04/10/2023, 7:28 PM

I'm having trouble finding the right seam to insert a deprecation of secondary ownership. Because realistically the only place we use in the core backends is

package

for Python for AWS Lambda and Google Cloud, but it could be used anywhere and where it is resolved is very far removed from where it is used 😐

proud-dentist-22844

04/10/2023, 11:02 PM

I was just looking at https://github.com/pantsbuild/pants/pull/18253 which documents a shortcoming in using a custom setup.py instead of a generated setup.py. Pants could generate an editable wheel from the

python_distribution

that provides its own

setup.py

and then extract the dependencies from the

.dist-info/METADATA

file. Or better yet, just run the PEP-517 backend method

prepare_metadata_for_build_wheel

(without everything else involved in building a PEP-660 editable wheel), and extract the requirements from there. It'd be a

python_requirements

target that gets its requirements from wheel metadata instead of defining the deps used to build the wheel. I know it's kind of a wild idea. How many people write their own

setup.py

? And would the ability to extract requirements from a

setup.py

be useful? -- Mayor of crazy town, signing off. 😜

happy-kitchen-89482

04/11/2023, 1:46 AM

Hypothesis: Python tools should not have default interpreter constraints. Instead, if their ICs aren't specified we should fall back to the ICs of the repo. Discussion in thread.

witty-family-13337

04/11/2023, 9:28 AM

Not sure if this was discussed already so apologies ahead if so. Considering an scenario where a

GenerateSourcesRequest

like:

Copy code

class GenerateMySourcesRequest(GenerateSourcesRequest):
    input = MySourceField
    output = MyGeneratedSourceField

Should

has_field(MyGeneratedSourceField)

return

True

when invoked in the target objects that have

MySourceField

as part of their core fields? Not really interesting on addressing the field of the generated sources, just trying to figure out a way of detecting targets that may generate code for a specific source field type.

bitter-ability-32190

04/12/2023, 7:30 PM

Waiting for mac runner has been a while: https://github.com/pantsbuild/pants/actions/runs/4680888321/jobs/8292814628?pr=18715

bitter-ability-32190

04/13/2023, 3:35 PM

Typo fix PR if anyone is bored and likes clicking green buttons: https://github.com/pantsbuild/pants/pull/18735

✅ 1

bitter-ability-32190

04/13/2023, 4:36 PM

FYI I think https://github.com/pantsbuild/pants/pull/18715 broke running Pants with Python 3.8:

Copy code

E   AttributeError: module 'hashlib' has no attribute '_Hash'

🚢 1

bitter-ability-32190

04/13/2023, 7:16 PM

Just to confirm my grep-foo isn't bad. Is there a GtiHub issue describing a feature request to have transitive 3rdparty deps as targets? https://github.com/pantsbuild/pants/issues/13283 is related, but not quite the same. I have a comment of a very real-world use-case that this would be a must-have for

✅ 1

sparse-lifeguard-95737

04/13/2023, 8:19 PM

I’m having a really tough time getting this cherry-pick through CI: https://github.com/pantsbuild/pants/pull/18734

src/python/pants/engine/streaming_workunit_handler_integration_test.py:streaming_workunit_handler_integration_test

is timing out even when every other test in the shard hits the remote cache. there are a few other tests timing out ~frequently but with enough retries they eventually seem to work

bitter-ability-32190

04/14/2023, 3:45 PM

Quick survey for y'all. Has anyone seen https://github.com/pantsbuild/pants/issues/18661 locally? Or only in CI?

Copy code

native_engine.IntrinsicError: Failed to create hardlink to /home/runner/.cache/pants/lmdb_store/immutable/files/<digest> at /tmp/pants-sandbox-O6nGOE/<path>: No such file or directory (os error 2)

proud-dentist-22844

04/15/2023, 12:21 AM

Where did the term

provides

come from in

python_distribution(provides=...)

? Do any other targets do something similar? I'm generating a config file for an

nfpm_package

(deb, rpm, apk), and I need to figure out how I want people to provide the details needed for it (

name

version

arch

depends

, ...). I could go with individual fields or a catch-all field like

provides

. edit: oh.
provides
has a special meaning for rpm/deb/etc so I'm not using that

happy-kitchen-89482

04/16/2023, 4:17 PM

Why does Lambdex require Python < 3.12? (https://github.com/pantsbuild/lambdex/blob/main/pyproject.toml#L34)

happy-kitchen-89482

04/17/2023, 10:58 PM

What is the status of the src/python/pants/backend/codegen/soap/java code? AFAICT there is no register.py that registers it? So I assume it's not in use anywhere? Should we delete it? cc @witty-family-13337

curved-television-6568

04/18/2023, 7:44 PM

@witty-crayon-22786 I’ve got a rule graph question about unions and in_scope_params 🧵

bitter-ability-32190

04/19/2023, 6:50 PM

Let's say someone added

RustPython

to our engine's deps. Now let's say someone wanted to rewrite our Python dep parser script in Rust. What do we think would yield the best perf in the average case? 1. An intrinsic which leverages the

local_cache

to cache Digest -> results ◦ Doesn't support remote cache 2. Build a full executable and use that 3. An intrinsic without caching I'm guessing 1 is enough because the code is likely faster than the latency of the network + overhead of the remote download code

proud-dentist-22844

04/19/2023, 8:04 PM

RFC on adding an

nfpm

backend that can generate

deb

rpm

apk

, and

archlinux

packages: https://docs.google.com/document/d/1b2ZvP_Ol6bOURHBaZYOD8gxixxHpV22S_zKiHKZw9iY/edit?usp=sharing

future-oxygen-10553

04/20/2023, 3:18 PM

I have a question about the docs for the `pex_binary.args` option. The annotation says that it should be an

Iterable[str]

, which from my experience with

subprocess.run

means it should be

["arg1", "arg2"]

, but this results in an error from Pex that the args should be a single string. I presume this behavior is intentional, but then does the

pex_binary.args

need to be an iterable? Could it just be a single string? If it needs to be an iterable for typing, I can try to update the docs

✅ 1

dry-analyst-73584

04/20/2023, 8:29 PM

Minor bikeshed: I've been working on an`audit` rule to check packages in lockfiles for vulnerability reports. Do you find the output readable? Any suggestions for improvements?

🙌 1

💥 3

happy-kitchen-89482

04/20/2023, 8:45 PM

Looks pretty readable

👍 1