Fishing for feedback on Python inferred string resources 🎣 🧵 https://github.com/pantsbuild/pants/pull/14049

@witty-crayon-22786 any ideas for how to determine whether we should check that

[scalac].plugins_global

exists or not?

@hundreds-father-404 (/ @ancient-vegetable-10556 ?) I'm trying to make sense of this after running

./build-support/bin/generate_all_lockfiles.sh

and after re-running after `rm -rf ~/.cache/pypoetry`:

$ git diff 3rdparty/python/requirements.txt
diff --git a/3rdparty/python/requirements.txt b/3rdparty/python/requirements.txt
index 61a0f5483..c4e115b90 100644
--- a/3rdparty/python/requirements.txt
+++ b/3rdparty/python/requirements.txt
@@ -14,7 +14,7 @@ humbug==0.2.7

 ijson==3.1.4
 packaging==21.3
-pex==2.1.61
+pex==2.1.64
 psutil==5.9.0
 pytest>=6.2.4,<6.3  # This should be kept in sync with `pytest.py`.
 PyYAML>=6.0,<7.0
^jsirois@gill ~/dev/pantsbuild/jsirois-pants (issues/14244 *) $ git diff 3^C
^jsirois@gill ~/dev/pantsbuild/jsirois-pants (issues/14244 *) $ git diff 3rdparty/python/requirements.txt 3rdparty/python/lockfiles/user_reqs.txt
diff --git a/3rdparty/python/lockfiles/user_reqs.txt b/3rdparty/python/lockfiles/user_reqs.txt
index c96827ddf..430d00428 100644
--- a/3rdparty/python/lockfiles/user_reqs.txt
+++ b/3rdparty/python/lockfiles/user_reqs.txt
@@ -11,13 +11,15 @@
 #   "generated_with_requirements": [
 #     "PyYAML<7.0,>=6.0",
 #     "ansicolors==1.1.8",
+#     "checksumdir==1.1.7",
 #     "chevron==0.14.0",
 #     "fasteners==0.16.3",
 #     "freezegun==1.1.0",
 #     "humbug==0.2.7",
 #     "ijson==3.1.4",
+#     "numpy==1.14.5",
 #     "packaging==21.3",
-#     "pex==2.1.61",
+#     "pex==2.1.64",
 #     "psutil==5.9.0",
 #     "pytest<6.3,>=6.2.4",
 #     "requests[security]>=2.25.1",
@@ -49,6 +51,8 @@ certifi==2021.10.8; python_version >= "2.7" and python_full_version < "3.0.0" or
...

Digging, but in case you've see something like this. I have no clue where in the heck numpy can be coming from. Certainly not Pex - its 0-dep.

here is a draft design for #13882: • https://docs.google.com/document/d/1mzZWnXiE6OkgMH_Dm7WeA3ck9veusQmr_c6GWPb_tsQ/edit feedback very welcome!

@witty-crayon-22786 (and anyone else!) re timing of 2.10 stable release & Python multiple (disjoint) lockfiles. I think within ~1 week we can ship it as a stable feature. Do we want to gate the release on this?

Can/should "resources" be renamed w.r.t. the union of

resources

and

files

? https://www.pantsbuild.org/docs/resources The ambiguity is killing me in my PR to infer

resources

from strings in Python 🪦 Can I propose

asset

?

Prep 2.10.0.dev3: https://github.com/pantsbuild/pants/pull/14302

I'm thinking folks will like this one: https://github.com/pantsbuild/pants/pull/14306 😄

I’m experimenting with a new builtin goal (builtin because I want to access the scheduler in ways incompatible when running from a rule). However, breaking out of the Python code when it’s not executing a rule seems a bit unclean.

In case the pants CI hits it (not sure if we use pip tools) https://twitter.com/hynek/status/1488019110151831552?s=21

I’m calling

graph_session.console.stdin

in a

BuiltinGoal

(context: for BSP support). I am getting this error:

Exception: Side-effects are not allowed in this context: SideEffecting types must be acquired via parameters to `@rule`s.

Is there a way to allow side effects with builtin goals? The

Console

is obtained from parameters to the

BuiltinGoal.run

method, and not as a

Param

to a rule.

@witty-crayon-22786 re https://github.com/pantsbuild/pants/pull/14309#issuecomment-1026105619 (

fmt

vs

fix

), I was hoping to stabilize Autoflake and PyUpgrade in 2.10 before this Friday, which requires resolving that discussion. But I think I should drop that goal, this isn't something to rush The other linter change I wanted to make for 2.10 is

lint --only

, which I think is safe to land

Hm @ancient-vegetable-10556 and @witty-crayon-22786, we never deprecated the first version of lockfile metadata for tool lockfiles, that uses

requirements_invalidation_digest

rather than

generated_with_requirements

./pants repl

is frustrating that it's hardcoded to default to

python

, even if you do

./pants repl jvm ::

Has anyone thought about compiling Pants with

mypyc

? 🤔

How do devs feel about https://github.com/typeddjango/pytest-mypy-plugins 🧵 ?

I’m looking at

RunRequest

and related infrastructure at the moment — is there a particular blocker around refactoring it to use a

Process

object rather than specifying raw args? Currently constructing a valid JVM process is complicated, so I’d love to make use of the

JvmProcess

infrastructure I used previously. (pinging @witty-crayon-22786 in particular since he’s seen the

JvmProcess

work thus far)

@witty-crayon-22786 the

--loop

flag does not seem to do anything when running in a builtin goal..?

wip… 😉

Failed to resolve requirements from PEX environment

after updating pytest's lockfile 🧵

Anyone have any thoughts on choosing one of these? • https://pypi.org/project/mypy-test/ -- would need to probably add as a plugin to

check

? So probably out. But like that it completely mirrors

mypy

• https://github.com/typeddjango/pytest-mypy-plugins -- I don't like the YAML personally • https://github.com/davidfritzsche/pytest-mypy-testing -- Seems easy to integrate, but would there be pitfalls when running with Pants?

Hi guys 👋 Would you mind releasing

pants-jupyter-plugin

with Python 3.10? 🙏 Assuming that only the version change is needed, if it's more than that, I can try to help. 😗

FYI I will fork 2.10 docs today & start updating them, given that we're close to rc0. Lmk if you want to make edits to 2.9 first

I'm following the instructions here (https://www.pantsbuild.org/docs/running-pants-from-sources#running-pants-from-sources-in-other-repos), I've curled the pants_from_sources, made it executable, ran it, and I have a mainline pants repo in a sibling repo - but regardless of anything I do (including trying other pants tags), I always get

ValueError: not enough values to unpack (expected 1, got 0)

for parse_requirements (full stacktrace in thread). Can that

pants_from_sources

be run (to test) without editing? Or do I need to fill out the backend_packages, pythonpath, and plugins for it to work?

@witty-crayon-22786 Can I get a sanity check on this? https://github.com/pantsbuild/pants/pull/14352

fyi @busy-vase-39202 the docs aren't showing a "submit" button for "Subscribe now" on https://www.pantsbuild.org/v2.10/docs/getting-help

A minor bikeshed opportunity 🎨 standardizing how we reference formatters, linters, and checkers. https://github.com/pantsbuild/pants/pull/14355. It will become part of our API once we have

lint --only

and friends

Is it assumed that any project using Pants, is also using git for version control? I’ve found myself a few times now wishing I could invoke pants from any directory, not just the root of the repo. And using

git rev-parse --show-toplevel

seems like a robust way of locating the repo root, in case of git repos at least. And in doing so, any specs could be treated as relative to where you invoked pants.

regarding parameterization: Eric’s comment yesterday highlighted that i had not considered

overrides

while finalizing the syntax with

@

first, followed by

#

. after thinking it through, i agree, and have updated the design doc to swap those back, and to add a section on the topic

@witty-crayon-22786 @fast-nail-55400 I'm writing docs + blog for Apache Thrift Python support. It's awesome, although looks like we haven't added Python->Thrift dep inference, meaning you need to explicitly add Thrift deps to

dependencies

on Python targets