https://pantsbuild.org/ logo
Join Slack
Powered by
Is there a known recipe for building a testing PEX...
# general
f
Is there a known recipe for building a testing PEX with 
pants package
? My issue is that Pants consumes precious precious memory that I need for my tests, so I'm thinking of doing something akin to
Copy code
pants package path/to/my:tests
pkill pantsd
./dist/path.to.my/tests.pex
c
I guess an interesting approach could be to have a “dry run” option that ony sets up the sandbox but doesn’t invoke the actual process, then going in and invoking 
__run.sh
from the sandbox..
(so to answer your question, not to my knowledge.. so I jumped straight to what feature could be invented to support this use case 😉 )
f
I'm thinking with tool resolves I could just add a pex_binary target that depends on pytest and has pytest:main as its entry point
c
maybe 🙂 worth a shot
f
yeah, i'll take a look
e
Wouldn't a 
pants export
do it? You could then run 
pytest
on your actual repo sources.
Basically like every other non Pants Python system for running tests.
f
I suppose that could work too. I like the convenience of being able to steal the single-file test pex though
Like, I could just ship that pex to another machine easily. Zipping up a venv and shipping it isn't impossible, but PEX is kinda that already with the tooling built in
e
Why would you ship tests to another machine?
I mean, obviously remote execution, but you're not solving that IIUC.
You know what your situation is. I just want to point out the simple obvious way to run tests in a Python project - do it from a venv. Like tox, nox, poetry, what have you.
f
Because I need my tests run on a single machine cloud emulator thingy that we control from our primary pipeline agent
In retrospect I would have put more effort into making the cloud emulator thing secure to use over the network, but it runs on the public internet and the cloud emulator server has 
sudo podman
permissions, so nobody felt really great about exposing that
e
Ok. I'm definitely confused since your nominal course of action is use Pants (
pants test ...
) but that uses too much mem; so I would think you could 
pants export && dist/venv/bin/pytest ...
instead (unless even 
pants export
takes too much mem). But your situation sounds maybe more complicated than I grasp.
f
It takes a few months to onboard to this monstrosity, no worries. Definitely none of it is how I would have done it if I had more say in the initial tech choices. But yeah basically I need to move tests to a remote machine (not the primary CI machine) where there exists a deployment of the software I want to run integration tests against (and that remote deployment is not something that can be managed easily in the Pants/REAPI model). So my thought is... just ship the tests as a pex. I suppose i could do 
pants export
on the remote machine before the tests start
e
Ok ... so your initial comment implying can't use 
pants test ...
- because too much mem use, was a bit misleading. Or maybe not.
f
I mean yeah there's that too. Pantsd has a tendency to use multiple GiB of memory and while I can limit it per invocation, if something requires it to resolve all deps it can eat a ton of memory (> 4 GiB) doing that, plus it takes time. I can spare the 4 GiB on the large remote machine where I send it to, but there's actually another execution mode where we deploy this stuff to the cloud and test against it with tests running locally on our primary CI runner, which only has 16 GiB
Open in Slack
PreviousNext
Powered by