Turns out I didn't have

pytest.ini

unignored in my

pants_ignore

in

pants.toml

. I'm confused how it could ever work though now 🤔

It was originally ignored cause we did a blanket policy and have been unignoring things as needed for incremental adoption purposes.

aha, yeah, I share your confusion

Is it possible to control the degree of parallelism that

test

runs with (ideally to 1, since I have problems with tests running in parallel that I will solve in the medium-term, but not in the short-term)

Feeling a touch foolish; we had some

python_requirements

in a top-level

BUILD

file which sourced a

constraints.txt

for the entire package, which had entries like

module_mapping={..."setuptools: ["pkg_resources"]...},
overrides={"torch": {"dependencies": [":setuptools"]}

so far so good (I know the pkg_resources should probably have been sorted automatically but we were having difficulties there). This used to work... but when I turn on

use_deprecated_python_macros=false

, it tells me I need to name that stanza of requirements. But if I give it a name (

name="root_requirements"

or some such) when I try to do anything like generate lockfiles I get

ResolveError: 'setuptools' was not found in namespace ''. Did you mean one of:

Okay, but if I try to name it (say, replace every instance of

setuptools

there with

//root_requirements:setuptools

) then I get

ResolveError: The file or directory 'root_requirements' does not exist on disk in the workspace, so the address 'root_requirements:setuptools' cannot be resolved.

... presumably because the requirements haven't been created yet or..? Just baffled; this worked fine earlier but presumably it was wrong then too. What's the right way to do this?

The address is probably

//:root_requirements#setuptools

//

starts at root.

:root_requirements

with colon selects that “generator target”.

#setuptools

selects one of its “generated targets”

Ah! Mostly. I'd seen reference to the hash # declarator but hadn't understood what it was. Let me try that

(Putting aside the question of why pants didn't infer the dependency.)

Was dependency inference at play? Seems like the issues stem from values provided in BUILD files? Unless you mean the

torch -. setuptools

issue, which is really just a prolific problem in Python packages, where they assume

setuptools

exists in every Python environment and so don't declare the dep

Probably the latter. Which yeah is not a Pants dep inference problem per se.

Generated targets are a bit new and more of a “under the hood” feature. So definitely something users may start to encounter more often and haven't seen before.

It's the latter for the bits in

overrides

. The one that threw me was that something in the past referred to

pkg_resources

but didn't pull in

setuptools

... but that was quite a while ago (these are legacy BUILD files by now!) I'll try removing that line and seeing if it's still necessary.

Awesome, tdyas, seems to have done the trick; thanks for explaining the generated target idea.

I love Python, but packaging is just so broken 😐

Are the generated lockfiles (3rdparty/python/default.lock) meant to be under version control in the same way that poetry lockfiles are, and manually regenerated when stated requirements change? (sorry, trying to understand the, er, dependencies of dependency lists. We were using a constraints.txt style "lockfile" which was a pip freeze of a venv with everything installed (including transitive 3rd-party dependencies we don't directly mention anywhere). Looks like Pants can now generate that full lockfile (and we could only generate

python_requirements

for our direct dependencies) but I wasn't sure of that yet)

• In Version control? Yes! (I think most ecosystem's lockfiles are in version control) • manually regenerated when stated requirements change? Yes-ish. If you use Pants to generate them, Pants will put a metadata header at the top, and if your inputs change Pants will warn/error.

Oh, but to clarify, do you mean 2.11.x JSON lockfiles, or the pre-2.11 txt lockfiles?

I'm on 2.10 now but could change to 2.11. Good to hear about the warning system.

If I'm making these updates, may as well try 2.11 I guess 🙂

The next RC should hopefully be 🚀 soon. The TL;DR is set

lockfile_generator

to

pex

and when you run

generate-lockfile

you'll get a PEX JSON. the PEX JSON contains transitive package metadata, so when Pants asks PEX to make pexs, it can do so fully in parallel, instead of semi-serially.

But wait for the next RC, it has a huge speedup in it 😉

(Well you can generate the lockfiles today, but consuming them is the thing that got fixed to be speedy)

See https://www.pantsbuild.org/v2.11/docs/reference-python#section-lockfile-generator

Awesome; really looking forward to it!

I did remove the line and things seeeeem to be going okay, so perhaps whatever required it was fixed some time in the past 🙂

Just checking; setting lockfile_generator in pants.toml in 2.11.0.dev2 and it's an error ("Invalid option 'lockfile_generator' under [python]"). Do I need to wait until the next RC? Was thrown since the documentation mentions it...

A quick question: if my python project depends on python39, can I use interpreter_constraints = [“>=3.7.*“] ? or the interpreter_constraints needs to align with the python dependency in the project?

Hi, can you clarify what you mean by "depends on python39"? Is this an explicit dependency? If so, how are you declaring it? Or do you just mean that your code uses Python 3.9 features?

use 2.11.0rc0 or later today rc1