https://pantsbuild.org/ logo
Join Slack
Powered by
Good evening! I asked some questions a few days ag...
# general
c
Good evening! I asked some questions a few days ago about how to use other version formats than setuptools_scm, and I got the good advice to use env(). This works fine for setting the version in python_artifact, but how can I do that for a pex binary? I tried to also generate a distribution, but that's not used for the pex file. Am I missing some configuration?
e
Probably. Does your 
pex_binary
target depend on the 
python_distribution
that uses 
env(...)
or does it depend on 
python_sources
? It needs to depend on the 
python_distribution
and only that - not the sources. You can 
zipinfo -1 my.pex
to find out. You should see 
.deps/*
entries for the 
python_distribution
wheel and no top-level source entries for things the wheel owns.
The issue with depending on both is the top-level source entry will be 1st on 
sys.path
and win. It won't see the version. Only the 
.deps/
wheel will see that, but it will never get imported.
Another debug tool:
Copy code
PEX_VERBOSE=1 PEX_INTERPRETER=1 ./my.pex -c 'import thing.that.lives.next.to.version as thing; print(thing.__file__)'
That will tell you both the sys.path the PEX set up and where it ultimately is loading your sources from - 1st party or wheel.
You might be able to get away with depending on both if and only if you use 
pex_binary(.., execution_mode="venv")
since, in a venv, all the code, 1st party and 3rd, gets installed in site-packages of the venv.
c
Excellent, thanks! I had it almost right, but couldn't see it because I had an underscore instead of a dash in the package name... Duh!
And now I will write my first macro to set all of this up in a simpler way!
If I make the pex file depend on the distribution, it looks like I also need to add dependencies on the libraries that this app uses. Shouldn't pants be able to infer the dependencies? Or does that work only for sources?
e
It should be able to. Let me find the issue(s). In short, depending on 
python_distribution
began it's life as a hack to support cython / C-extensions by letting setup.py deal with that. And the hack has never been revisited comprehensively, only moles have been whacked.
That said - the two things to ask are - why do you need the version number so badly? And, given you probably have a good reason - there are 18 ways to get it.
c
I see, much appreciated. The thing is that I might do weird stuff, and I don't want to send you away chasing ghosts, only to discover I forgot a comma or something :-)
e
c
I want to have the version number because we had some trouble with people installing/deploying wrong version and chasing bugs that had already been fixed in the right version. But we couldn't know which version they had.
e
That makes sense for a published 
python_distribution
- it does not make as much sense for a 
pex_binary
. The version a Pex uses in Pants is the sources as they are right now in the repo!
The version is ~the git sha.
c
This python distribution thing seems like it is a can of worms. What I really would be happy to have is a vcs_version that can get the version from another source. Or if the version would be part of the pex name, just like for the wheel.
e
Well you already have that solution staring at you.
You can 
env
parametrize any field in a build, save for 
name
for sanity sake, so best read up on the fields.
c
Oh, so I can specify the file name too, not only the path?
e
Yes.
c
🕺🏽
Thanks a lot, this will be good enough!
e
So that should cover things I think. In 2.16.+ you get these two new fields: https://github.com/pantsbuild/pants/blob/34d0157c251692089d8e8fe8a077c7059a0502c2/src/python/pants/backend/python/target_types.py#L380-L397 So you could embed the version via 
env
in a PEX permanent env var your PEXed code could read - say 
VERSION
.
c
Even better! BTW, would it be difficult to make available the 2.16 docs online?
e
We have a standard procedure and pace for the online docs: https://www.pantsbuild.org/docs/release-process#step-2-update-this-docs-site It looks like this was not followed for 2.16.0.dev0. I think we're inconsistent in what we do for .devX vs what the docs say - let me check in #development. That said, docs are over-rated:
Copy code
$ git diff
diff --git a/pants.toml b/pants.toml
index 0cecdef..7715879 100644
--- a/pants.toml
+++ b/pants.toml
@@ -1,5 +1,5 @@
 [GLOBAL]
-pants_version = "2.15.0"
+pants_version = "2.16.0.dev7"

 backend_packages = [
     "pants.backend.python",
And then:
Copy code
$ pants help pex_binary

`pex_binary` target
-------------------

A Python target that can be converted into an executable PEX file.

PEX files are self-contained executable files that contain a complete Python environment capable of running the target. For more information, see <https://www.pantsbuild.org/v2.16/docs/pex-files>.


Activated by pants.backend.python
Valid fields:

args
    type: Iterable[str] | None
    default: None

    Freeze these command-line args into the PEX. Allows you to run generic entry points on specific arguments without creating a shim file.

...

env
    type: Dict[str, str] | None
    default: None

    Freeze these environment variables into the PEX. Allows you to run generic entry points on a specific environment without creating a shim file.

...
c
Yes, but using the cli requires to know what I am looking for. How would I dream that there are some new fields that do what I need? Having text search helps finding info in unexpected places, or there is an overview page. Of course, for a dev release, all is not in place, but imho it's still better.
e
Well, we can agree to disagree there - but most people would definitely agree with you. I've raised the issue here: https://pantsbuild.slack.com/archives/C0D7TNJHL/p1677881297276749 We have a 2.16.0a0 release this weekend; so I suspect we'll have a doc site by ~Monday either way and hopefully more clarity about why / when we cut them.
👍🏽 1
5 Views
Open in Slack
PreviousNext
Powered by