Hi all--I'm hoping to produce a self-contained executable for a CLI, built using Python. With the abandonment of PyOxidizer (and therefore, the non-option for me of using the pyoxidizer_binary target), this note from the "Installing Pants" docs made me curious: are there still active plans to do the following?

...soon add support in Pants for building scies out of your code, which will allow you to package and ship fully standalone Python binaries!

Yes. This is strictly a time thing for me: https://github.com/sureshjoshi/pants-plugins/tree/main/pants-plugins/experimental/scie I use this often, and I love it: Public example: https://github.com/sureshjoshi/pantsible

Oh interesting, great to know, thank you @wide-midnight-78598! I spent a bit of time trying to understand

scie

but ultimately think it's a bit beyond me. So, I'm actually not fully certain I understand the nuance...would the one file that results from the

scie_binary

target expected to be executable on major, up-to-date operating systems (macos, ubuntu, windows; assuming

platforms

is set accordingly)? Are there implicit dependencies on other tools/programs that absolutely must exist on the target host for the

scie

to execute?

The readme here is probably the best, short explanation for it: https://github.com/a-scie/jump Handwavy example: Using

pantsible

, that's ansible put into an executable pex file (not a requirement, just makes my life easier), merged with Python Build Standalone (https://github.com/indygreg/python-build-standalone) for the Python interpreter, and then a small shim that orchestrates the unzipping, and calls, and whatnot (there is also a JSON manifest which contains the instructions on how everything interacts) So, I get an executable file, that I can copy, put in my path, whatever. Upon execution, based on the instructions I gave it, it knows to download Python 3.12 and store that in cache, and then to unpack and pre-compile ansible into a venv, and then it knows what commands to call on my ansible venv I can't say for certain if there are specific tools/programs, but I think it would be at the level of... glibc or musl or whatever - but I think the shimmed scie code otherwise runs standalone. I haven't looked into whether the stuff that needs to be on the OS is due to Python, or scie - to be honest. Assuming you're running on mac or linux, if you setup your platforms accordingly, you should be good to go. In my case, I'm using a pex, and pex doesn't run on Windows What's your use case?

pantsible

works for me (and is a cool idea)! I think this is starting to click. So: the

pex

is installed in a venv, in a Python interpreter that is downloaded from Python Build Standalone on first run, and arg0 given to the

scie

is mapped to an executable exposed by the

pex

(which are really just binary scripts in the venv;

venv/bin/...

).

In my case, I'm using a pex, and pex doesn't run on Windows

Hm, yeah, interesting. I'd love to build a windows distribute-able binary as well, in addition to linux and mac.

What's your use case?

The general use case is a CLI application for a SaaS data platform (https://www.roboto.ai/). Our customers use all three platforms (linux, macos, windows). I've put some effort into producing a standalone binary with PyInstaller (I saw a reference in the Pants Github issues history to an attempted

pyinstaller_binary

Pants target a while back...), but using PyInstaller complicates our CI, and the cold-start times of a PyInstaller binary on macOS is atrocious until we code sign the artifact.

So, one of the things that a recent update to pantsible does is that it uses a `scie`’s ability to be a busy-box, which is how we can use all of those ansible commands (e.g. ansible-vault, ansible-playbook). I’ve used similar functionality elsewhere, for example, to kick off a server - but also have the ability to run a

SCIE_BOOT=migrate ./apigateway

type of thing - so multiple commands. It’s been a long time since I’ve used anything but pex, so I can’t speak to shiv. I haven’t given it much thought - but maybe you can even just use “loose” python files that are unrelatedly packages (zipped or tarred or something). In the

scie

plugin, I specifically used pex because it’s built into pants and I’ve used them so much otherwise, but I can’t really suggest that it’s a requirement per se. I’m being a bit vague, because normally I would sanity test something on my machine before giving it a yay or nay, but I’m not near my machine at the moment. Otherwise, I’d probably try to whip something up with a custom scie that points the interpreter to a main.py and see what happens (entirely outside of pants for the sanity). If you look at the docs, examples, and tests on a-scie/Lift or Jump - you can see some other reference examples - like using

cat

to create a scie, or using node. To me that would imply that using other packaging mechanisms would work

Thank you @wide-midnight-78598!

@strong-dawn-11939 Just a heads up that pex is getting native functionality to bundle a python interpreter with the pex as we speak, so this might become even more streamlined in the future - if your goal is to strictly run your pex with a bundled interpreter (not like

pantisble

where I added the busybox functionality and stuff)

Ooo, yes! Is there a Github PR that I can follow?

https://github.com/pex-tool/pex/releases/tag/v2.11.0 It's released, not yet in Pants - as that might take some time/effort. This is the ticket to port my scie plugin over, but I think now that will take some more thought, as we probably want to use pex directly for simple cases, and science for harder cases... Maybe https://github.com/pantsbuild/pants/issues/21100

But that doesn't feel right either - so maybe just a regular scie_target that internally decides what it wants to do

Understanding only a fraction of this, assuming

pex --scie ...

produces the same type of output as

scie_target

, then, I'd agree with this!