https://pantsbuild.org/ logo
#general
Title
# general
b

bulky-mouse-48830

09/20/2022, 2:20 PM
It seems like a very basic question, but I can't find any way to override the
target-system
for lockfile generation. I am only interested in
linux
as a target system, but Pants seems to fail because it wants dependencies to be resolved for all platforms.
βœ… 1
c

curved-television-6568

09/20/2022, 2:31 PM
If using pex as lockfile generator, this is currently hard wired: https://github.com/pantsbuild/pants/blob/0bbf1fbb6078cacb1e3a75f5ea67a14d0398910c/src/python/pants/backend/python/goals/lockfile.py#L153-L164 Not sure how this is if using poetry..
As the comment there hints at, this could likely make sense to be configurable πŸ˜‰
b

bulky-mouse-48830

09/20/2022, 2:33 PM
I use pip-tools currently. Works perfectly. It probably makes sense for libraries to be universal. But for applications it doesn't make sense. My application only runs on linux in containers.
c

curved-television-6568

09/20/2022, 2:33 PM
But I wonder if you can’t add environment markers to your requirement that is causing issues?
b

bulky-mouse-48830

09/20/2022, 2:34 PM
How do I do that?
c

curved-television-6568

09/20/2022, 2:35 PM
This is the relevant PEP: https://peps.python.org/pep-0496/ So you can use that in your requirements.txt file for instance.
b

bulky-mouse-48830

09/20/2022, 2:36 PM
Thanks. I'll see if I can make it work that way
c

curved-television-6568

09/20/2022, 2:36 PM
Sorry, got the rejected version, use this one https://peps.python.org/pep-0508/
e

enough-analyst-54434

09/20/2022, 2:41 PM
@bulky-mouse-48830 if you'd like to contribute a fix to un-hardwire this, I'd be grateful. If even just filing a feature request issue.
b

bulky-mouse-48830

09/20/2022, 2:41 PM
I will try. Thanks
e

enough-analyst-54434

09/20/2022, 2:42 PM
FWIW, the feature was only added to work around similar issues to yours, but for Windows artifacts. Since neither Pants nor Pex currently support Windows, Pants hardwires just Linux and Mac now, whereas before the
--target-system
was added to Pex, it was always all 3!
Pex also supports pip-compile-style strict locks - just lock for the interpreter in the venv pip-compile is installed in - but Pants has never used this locking mode - it hardcodes
--style universal
; so the un-hard-coding could go in a few directions.
b

bulky-mouse-48830

09/20/2022, 2:45 PM
Ah, that'd be nice. I already have pip-compile lock files.
e

enough-analyst-54434

09/20/2022, 2:46 PM
By style I just mean the strict spirit of exactly the artifacts used in this venv, the format is still that same json format, the contents though are restricted to the exact set of artifacts used and no more.
βœ… 1
You use
pex3 lock export
to get out pip-compile format requirements.txt file with hashes.
Basically there are alot of Pex lock tools, Pants just exposes one narrow use of them.
πŸ‘ 1
πŸ™ 1
h

hundreds-father-404

09/20/2022, 3:44 PM
+1 to this being an option.
It probably makes sense for libraries to be universal. But for applications it doesn't make sense.
@bulky-mouse-48830 this is an important thing for us to figure out: is it worth having the flexibility to set the target platform "per-resolve" (aka lockfile)? Vs an option that applies to all lockfiles is sufficient. We have all the infrastructure for per-resolve option, and it would look like the
resolves_to_constraints_file
option. So it's easy to pull off. Only a little more complex of an API for end users and more complex pantsbuild/pants code FYI with the per-resolve option, there is a way to set the same value for every single lockfile, and then override specific ones
b

bulky-mouse-48830

09/20/2022, 4:16 PM
@hundreds-father-404 I'm sorry, I'm still very much a newbie to Pants and I do not understand the internals at all.
h

hundreds-father-404

09/20/2022, 4:19 PM
No worries, your perspective is particularly valuable for that reason! So, Pants allows you to have multiple "resolves", which are logical names for lockfiles. For your own user code, you can define as many as you want: maybe you have a data science project that should have zero impact on the dependencies used by your web app project. There are also "resolves"/lockfiles for each tool that Pants runs, like Black and Pytest I'm trying to confirm that it's worth the bump in complexity to let users set the target platform differently for each resolve/lockfile. For example, for data science, I know the team only uses Linux so I'm going to target that. But our web app team loves macOS, so we need both Linux and macOS Does that make sense?
b

bulky-mouse-48830

09/20/2022, 4:22 PM
Makes sense.
πŸ‘ 1
Especially in a large monorepo, I can see per-resolve target-system being useful. But that's not my specific use case. For me one target-system that works flawlessly is enough.
πŸ‘ 1