https://pantsbuild.slack.com/archives/CH9PU7A2K/p1574847983000200 thanks so much for all the feedback on this! let types run rampant over the codebase!

Seeking feedback on this issue: https://github.com/pantsbuild/pants/issues/8727

... which is not inlining its description for some reason

Design: python requirement constraints (AKA lockfiles) #8727

this is a really silly question — what does the constraints file provide over just using requirements? i know this is an existing python idiom, but i’m still trying to understand the utility of that. if we wanted versions to be constrained, can’t we just convert all our requirements to == by hand?

Not a silly question! See https://www.kennethreitz.org/essays/a-better-pip-workflow for why Pipenv argues that maintaining an exact

requirements.txt

is not ideal

A) You'd have to maintain, by hand, not just the requirements that you care about but those of the transitive closure of dependencies. B) How would you make those decisions? You need a resolver to find a valid combination for you. That is in general an NP-hard problem. Best to leave it to an algorithm than a human...

Ok, but why not let the resolver do it for you once, then put that in your

requirements.txt

instead of this new constraints file?

also I believe P = NP but can't prove it yet

also the pipenv site has an SSL error but do not fret i can google

Because then you lose the distinction between "things I actually want to depend on, and constraints I impose as a human" vs. "the entire transitive closure of those things, at one specific version each".

For example, say you want to upgrade just one thing. You can't do that in the lockfile, because it may no longer be compatible with the other constraints. You need that original

requirements.txt

as the source from which a new lockfile can be generated.

this makes sense and is really interesting

i'm wondering whether this could be relevant to the dehydrated/ipex idea in #pex, which requires a transitive resolve anyway. if we were to enforce in pants that every

python_binary()

target that we want to generate an ipex for must have an associated lockfile, that would mean we could avoid any need to modify pex to introduce the

--dehydrated

flag, for example.

i think that's a really interesting idea and will make a comment to that effect in the gdoc

cc @average-vr-56795 ^ this might be a good effort to piggyback off of to avoid having to worry about doing anything special for the resolve part of ipex

@aloof-angle-91616 I’m not sure this changes anything there? Fundamentally, there are two relevant files for a target:

target-specific subset of requirements.txt

(“unresolved deps I care about for this target”) and some kind of lockfile (“resolved transitive deps I will actually use for this target”) This proposal is just that we’ll store a third, non-target-specific file (“resolved superset of deps I may use”) of which target-specific lockfiles are guaranteed to be a subset? Basically just that we’re caching the result of a global resolve to constrain our future resolves, for consistency. Or am I mis-interpreting the proposal?

Assuming that’s right, we’d still need to modify

pex

both because we’d need a per-target lockfile, and because we’d need something to actually do the fetch on bootstrap. The only thing that would change is that if you built the

pex

twice in a row, clearing caches in between times, you’re more likely to get the same result, because we’re effectively caching part of the resolve in a new place.

I guess if our constraints file had enough structure in it to note dependency relationships between requirements, it could effectively cache the entire resolve, but that’s a pretty different project.

I’m going to make a PR which changes our zinc wrapper, and therefore needs a new version of zinc to be published. Should I post the PR without publishing the new zinc version to maven in case there are changes requested? It feels a bit like a chicken and egg situation, advice appreciated.

hope everyone who works on pants has had an enjoyable thanksgiving, or an enjoyable past several days as the case may be 🙂

is there up-to-date documentation on how to add a subsystem in a plugin anywhere?

Hi all, have a question. Was looking at the PR merged in for black support assuming it would be available under the lint goal. After digging through some threads, it seems like this is a part of a larger effort of a v2 feature set. Attempted to run

./pants lint-v2 <target>

on

1.23.0rc0

and ran into an error:

ERROR: Not a registered union type: <class 'pants.engine.rules.TargetWithSources'>

Which led me here: https://github.com/pantsbuild/pants/pull/8490. And I figured I was heading deeper down the rabbit hole. So question is, are v2 goals usable outside of pants development itself? and if so what's the uplift of bringing them into our project

can the

pants_version

global option be configured to fetch the latest version of pants/master ?

Does anyone know why https://github.com/pantsbuild/pants/blob/master/src/python/pants/backend/project_info/tasks/idea_plugin_gen.py uses

self.context.options.positional_args

rather than

[target.address.reference() for target in self.context.target_roots()]

or similar?

would anyone object if I combined the separate

write_stdout

and

print_stdout

methods on

Console

(and their analogous

_stderr

variants) into a single method?

it seems kind of redundant to have both

in the vein of writing things to stdout/stderr, i remembered an idea i had a while back on how to make it easier to do things like parse compiler error output from pants without having to parse the entire pants output to find the compiler stderr in the first place, and realized that we could also make it possible to tell pants to show all the output from all the subprocesses, even if they succeeded. the result is this

--workunit-output-redirections

arg in this PR which tells pants how to redirect output sent to matching workunits, and i'd love for any pants user to weigh in on this proposed dict option interface: https://github.com/pantsbuild/pants/pull/8765

Hi @hundreds-father-404 Daniel said that you may help I get error when run python integration test. The command is:

./pants --v2 --no-v1 test tests/python/pants_test/bin:integration