Is it possible to create a single python (pex based) lock file with multiple versions of the same package? I have a small number of components that need to support a broad range of interpreter versions (going as low as 3.6), and it’s impossible to get all our 3rd party deps to support this far. I’ve had success isolating those components to their own resolve/lock-file, but I feel like I’m losing one of the core benefits of the monorepo since I now need to carefully self manage which resolve is used and which components share code. Is there a better way?

I'm very interested in this as well, specifically for

pytorch

.

Yes, but Pants doesn't use this style of lock file. The pex3 lock create command supports

--style {strict, sources, universal}

and Pants always uses

universal

today. The universal mode creates 1 lock that must work for the complete range of interpreters and machines implied by

--interpreter-constraint

and any

--target-system

specified (Pants always passes

--target-system mac --target-system linux

). The strict and sources modes, on the other hand, use

--interpreter-constraint

on the local machine to find all interpreters meeting those constraints and then, for each interpreter, creates a lock and all of these locks are stored in the one lock file. When the lock is later consumed to produce a PEX for a specific interpreter, the best-fit lock is selected from the lock file. The only difference between strict and sources is that strict locks exactly what each selected interpreter needs and no more whereas sources also locks the sdist associated with each wheel locked if there is an sdist. A challenge using these modes is to ensure the machine you create the lock on has all the interpreters needed installed. Otherwise, if you have

--interpreter-constraint ">3.7,<3.10"

and you create the strict or sources lock on a machine with only Python 3.8, the lock file will only contain a single lock for the Python 3.8 interpreter but not 3.7, 3.9 or 3.10 (and even that is simplistic since real artifact in the wild have constraints like `Requires-Python >=3.7.2`; so to cover all bases you really need every minor release of Python too unless you know your resolve real well and know this is not a factor. With universal mode, the lock create command can safely be run on any machine.

I'm not sure I follow what you mean. Also just to make sure we're all on the same page, we're talking about different versions of the same package. E.g pytorch 1.2.2 AND pytorch 1.2.2+cu102

We're on the same page. In plain old non Pants you'd have 2 requirements:

pytorch==1.2.2; <marker1>
pytorch==1.2.2+_cu102; <marker2>

You'd arrange marker1 and marker2 to be mutually exclusive.

Yeah, same execution environment in either case. Just wanting to toggle between reqs when told

Yeah, Pex / Pip / PyPA can't help you there. That would have to come from Pants or higher.

Mxnet got this right. They use a different package name but same module name

That's bad too! Now you can have both packages on sys.path and you'll get the import from a ~random one!

Oh I agree it's nasty. But at least it's workable. I'm kneecapped with pytorch