pantsbuild #development

, we know that the file in question has no 3rd party deps, then skip resolving `constraints.txt`/`lockfile.txt` might make our code more complex, but seems justified? Will also help w/ incremental migrations, that you don't need your 3rd party deps figured out for some tests/scripts/binaries

🔥 1

average-vr-56795

08/02/2021, 9:45 PM

Why do they currently get resolved if nothing in the build depends on them?

average-vr-56795

08/02/2021, 9:46 PM

Surely they shouldn't being reachable from the relevant roots of the graph?

hundreds-father-404

08/02/2021, 9:54 PM

See "Fine-grained resolves without a performance hit" in https://blog.pantsbuild.org/introducing-pants-2-5/ for an explanation The optimization is great normally, but unnecessary if 3rdparty deps aren't in use

witty-crayon-22786

08/02/2021, 9:54 PM

yea, i would think that they would be skipped. maybe we don’t filter for the empty set case?

hundreds-father-404

08/02/2021, 9:55 PM

maybe we don’t filter for the empty set case?

We do not

enough-analyst-54434

08/02/2021, 9:58 PM

I mean, this seems like an obviously good thing to do with no downsides right?

👍 1

enough-analyst-54434

08/02/2021, 10:00 PM

That said, the use case does sound machine driven. Is this what happens when you 1st tailor and then try to do something? Whereas if you didn't tailor, but instead left off declaring a

python_requirements()

or similar target and just hacked on the dependcy-less scripts - you'd not hit it.

enough-analyst-54434

08/02/2021, 10:01 PM

Even if contrived, sgtm.

hundreds-father-404

08/02/2021, 10:02 PM

Currently we don't add

python_requirements

./pants tailor

, I think only b/c it uses the old school macro and that's not wired up yet So, yeah, for a brand new repo, you'd only hit this if you have added

python_requirements()

in between running

./pants tailor

and trying your first

./pants test

etc

hundreds-father-404

08/02/2021, 10:10 PM

For example-python repo, I got it working on my M1 but only by changing

[python-setup].interpreter_constraints

to be >=3.9, rather than >=3.7 I imagine 3.9 isn't popular enough yet for us to change the default for everyone? Next best thing is add a note to the

pants.toml

and README.md?

average-vr-56795

08/02/2021, 10:25 PM

It sounds like the current implementation is the more contrived one? It sounds like currently there's a hard-coded "always do resolves", and the more automated driving would be for the partial-sub-resolve to be a node derived from something resembling a singleton, and that by not requesting any nodes that happen to be derived from the singleton, nothing drives the singleton to be computed?

happy-kitchen-89482

08/02/2021, 11:47 PM

Yeah, I would have expected no resolve to happen if no sub-part of it was required. Why does it happen?

hundreds-father-404

08/03/2021, 2:06 AM

might make our code more complex

lol, 2 line change. https://github.com/pantsbuild/pants/pull/12488 Sorry for the noise...this change was not in any way controversial and didn't really warrant a dedicated thread

➕ 1

🎉 1

hundreds-father-404

08/03/2021, 4:14 AM

Another performance idea. In a couple places, we have code to robustly handle mixed interpreter constraints in your project, like some Py2 only code and some Py3 only code. We do things like: partition Flake8 so that it runs on each distinct partition w/ the correct interpreter constraints But if you are in a single interpreter-constraints repo, then this is an unnecessary hit to perf, no matter how much we optimize. So, idea: add

[python-setup].allow_mixed_interpreter_constraints

. If false, ban the

interpreter_constraints

field and solely allow for the global

[python-setup].interpreter_constraints

Thoughts?

hundreds-father-404

08/03/2021, 4:15 AM

The main motivation is how slow it is to compute interpreter constraints for tool lockfiles: https://github.com/pantsbuild/pants/issues/12415#issuecomment-891504766 It requires consulting the entire repository to inspect what you're using for tools like Pytest and Flake8, which is slow and prone to frequent cache invalidation

hundreds-father-404

08/03/2021, 4:19 AM

I think that we would default to False. You have to opt in to multiple interpreter constraints working in your repo That makes sense to me, this is an advanced feature many orgs can ignore. Default to doing the more performant thing

hundreds-father-404

08/03/2021, 3:44 PM

Even better imo is to require activating pants.backed.python.mixed_interpreter_constraints, which would allow us to conditionally register the interpreter_constraints field so it doesn't show up in

./pants help

otherwise. Semantically, you're opting into that advanced functionality by enabling the plugin

witty-crayon-22786

08/03/2021, 4:38 PM

it’d be great not to need that flag, but yea. challenging.

witty-crayon-22786

08/03/2021, 4:39 PM

one middle ground might be to over-resolve slightly, and rather than actually compute the precise reachable constraints from pytest targets, just compute all interpreter constraints in the repo.

witty-crayon-22786

08/03/2021, 4:39 PM

even though some of them might not be reachable.