thereby avoiding the download of e.g. gigabyte size wheels

cc @enough-analyst-54434 @hundreds-father-404 @witty-crayon-22786

seeing this warning on latest main branch:

22:33:26.91 [INFO] Scheduler initialized.
22:33:27.77 [WARN] Unmatched globs from src/python/pants/backend/codegen/thrift/scrooge:tests's `sources` field: ["src/python/pants/backend/codegen/thrift/scrooge/*_test.py", "src/python/pants/backend/codegen/thrift/scrooge/test_*.py", "src/python/pants/backend/codegen/thrift/scrooge/tests.py"]

Thought dumping before the long weekend, sorry if it isn't fully formed. I thought of a possible performance improvement that sits between linting the world and linting process-per-file. Process per dependency set. E.g. if 10 files have depset A, and 12 files had depset B, then you'd lint in 2 processes. I'm wondering what the performance characteristics would be in a real-world scenario when compared to the current 2 options. (I think this is different from the batching proposal as well, but in the same vain)

Tangentially related, is there a good reason format-process-per-file isn't the default? Since formatting doesn't care about dependencies it should almost always be the faster choice on a multi core machine

E         	  in pants.core.goals.check.check
E         	  in pants.backend.python.typecheck.mypy.rules.mypy_typecheck
E         	  in pants.backend.python.typecheck.mypy.rules.mypy_typecheck_partition
E         	  in pants.backend.python.util_rules.pex.create_pex
E         	  in pants.backend.python.util_rules.pex.build_pex (requirements.pex)
E         	  in pants.engine.process.fallible_to_exec_result_or_raise
E         	Traceback (most recent call last):
E         	  File "/tmp/process-executionIAlw9H/src/python/pants/engine/process.py", line 282, in fallible_to_exec_result_or_raise
E         	    raise ProcessExecutionFailure(
E         	pants.engine.process.ProcessExecutionFailure: Process 'Building requirements.pex with 2 requirements: pantsbuild.pants.testutil@ file:///tmp/process-executionIAlw9H/pantsbuild.pants.testutil-2.10.0.dev0-py37.py38.py39-none-any.whl, pantsbuild.pants@ file:///tmp/process-executionIAlw9H/pantsbuild.pants-2.10.0.dev0-cp39-cp39-linux_x86_64.whl' failed with exit code 1.
E         	stdout:
E         	
E         	stderr:
E         	ERROR: pantsbuild.pants-2.10.0.dev0-cp39-cp39-linux_x86_64.whl is not a supported wheel on this platform.
E         	pid 2661 -> /home/runner/.cache/pants/named_caches/pex_root/venvs/92e3acc3a38b2d01c1bdea319c3ee88b6bbd73af/2703f860957c2efe7f4d087e27e538da0891bc96/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/runner/.cache/pants/named_caches/pex_root --log /tmp/process-executionMDFAjN/.tmp/tmp8vrb9lf9/pip.log download --dest /tmp/process-executionMDFAjN/.tmp/tmp_qlwclys/opt.hostedtoolcache.Python.3.7.12.x64.bin.python3.7 pantsbuild.pants.testutil@ file:///tmp/process-executionIAlw9H/pantsbuild.pants.testutil-2.10.0.dev0-py37.py38.py39-none-any.whl pantsbuild.pants@ file:///tmp/process-executionIAlw9H/pantsbuild.pants-2.10.0.dev0-cp39-cp39-linux_x86_64.whl --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
E         	None
E

From https://github.com/pantsbuild/pants/runs/4678547970?check_suite_focus=true#step:14:493 feels like an issue with how we package pants.

Why does

importlib.import_module("a.module")

go in the "string imports" bucket, but

__import__("a.module")

not?

On this page: https://www.pantsbuild.org/docs/resources The middle tab (

src/python/project/BUILD

) uses

resource

(singular) with

sources

(plural)

How does the Pants devs feel about pragmas? Specifically, I'm having an issue where an import should be ignored (but I very much like having

unowned-dependency-behavior

on). I'm thinking it'd be really convenient to do:

import foo  # pants: ignore

On this page: the output of

./pants lint

has what appears to be a "can't-decode-character" symbol a few times. Is that expected or is that due to coloring/rendering? https://www.pantsbuild.org/docs/shell

Also FYI I'm getting bootstrap issues on

main

😞

(IGNORE)

@happy-kitchen-89482, @hundreds-father-404, @ancient-vegetable-10556, @enough-analyst-54434: i think that last time https://github.com/pantsbuild/pants/pull/14050 occurred (plugin with an incompatible requirement), i had hoped that moving to releasing a PEX (via #12397) would prevent the issue from recurring. but thinking about it more, anything that doesn’t do an on-the fly resolve at use-time (PEX, a lockfile, a rust binary, etc) would all potentially still suffer from this issue

It seems that there’s no version

2.9.0rc2

for macos arm64, there was for

rc0

Anyone having difficulty bootstrapping on mac os on main today?

Why can't

lint-per-file-process

use

repository.pex

directly?

💥

joshuacannon@CEPHANDRIUS:~/work/pants$ set RUST_BACKTRACE=1
joshuacannon@CEPHANDRIUS:~/work/pants$ ./pants peek src/python/pants/backend/python/dependency_inference/scripts/BUILD
06:36:15.38 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'TypeError'>, value: TypeError("'int' object cannot be converted to 'PyString'"), traceback: None }', src/externs/mod.rs:145
06:36:15.38 [ERROR] Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.

Friendly ping to get feedback on the direction of a new

BuiltinGoal

type: https://github.com/pantsbuild/pants/pull/13991 cc @witty-crayon-22786? 😉

on the topic of packaging a pants pex and distributing pants as a pex (https://github.com/pantsbuild/pants/issues/12397) would that mean that we could have a “pants binary” installed on the system rather than relying on a “bootstrap”

./pants

script in every repo?

planning to cut another 2.9.0 rc today. any other changes inbound? nothing is marked

hey folks. https://www.pantsbuild.org/v2.9/docs/reference-update-build-files

update-build-files

is great because

fmt

will format your

BUILD

files code. However, I wonder how difficult would it be to extend the goal to let user pick a formatter (currently only

black

is supported). The code of interest: https://github.com/pantsbuild/pants/blob/d3b1d24372a60aca0cbd3c54b73363ddfac4b883/src/python/pants/core/goals/update_build_files.py#L258-L257. It would be helpful to be able to choose a formatter for the

BUILD

files. Is it something trivial or would it require a deeper thought?

hey folks! have some very rough draft notes for a 2.9.x blog post to go out this week: if you see anything missing from the list of highlighted features, please comment/let-me-know: https://docs.google.com/document/d/1_O1ljAtOytl6i-ulY8aiWDplilzljIB_ESsqoRQNp6Y/edit#

did i miss any outstanding performance issues? https://github.com/orgs/pantsbuild/projects/2/views/1

last week I added a depandabot config to automatically create PRs to update rust crates: https://github.com/pantsbuild/pants/blob/main/.github/dependabot.yml However this doesn't seem to work, for example when I added this file in our private repo we got a depandabot tab under

Dependency graph

page: https://github.com/pantsbuild/pants/network/dependencies which makes me thing that GH ignores the file the docs says someone with a write access need to modify it and maybe because I don't have that access my change is ignored... https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependen[…]tomatically/enabling-and-disabling-dependabot-version-updates @hundreds-father-404 can u try and make a trivial change to this file and see if that helps (maybe change the schedule to daily updates and tweak the hour to noon PST): https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependen[…]ed-automatically/configuration-options-for-dependency-updates

I have a very strange issue, python test, that hangs if I run it under pants, but passes if I run it with

--debug

. Have you heard about something like this?

This hangs:

./pants test ./test_queue.py

This passes:

./pants test --debug ./test_queue.py

A survey of where we are with targets. Lots of insights the past two years + a few open questions still. https://docs.google.com/document/d/1Xx5Vq-3Zl-uGy6Lkghg23ZriWpEsAmPAjHxau-pq-Bo/edit# Comments welcomed, including if you think there are issues with targets that we're not yet capturing

If I specify an invalid file in a

python_source

target, I get a rather misleading error 🤔

pants.engine.target.InvalidFieldException: The 'source' field in target src/to/target:the_target must have 1 file, but it had 0 files.

I’m about to factor some code out of

backend/python

into something that will support code in both

backend/python

and

backend/jvm

— is there an OK place to put this? I see

util

, which seems logically far away. Would we be ok with a new

backend/common

directory?

@polite-garden-50641: once the initial flood of bumps comes through, will dependabot mostly send out PRs according to its schedule? i.e., on thursday?