https://pantsbuild.org/ logo
#general
Title
# general
p

powerful-florist-1807

05/14/2021, 1:00 AM
Hey there, is there any example on building and deploying PEX to AWS EC2 for execution?
e

enough-analyst-54434

05/14/2021, 1:16 AM
Not really. Which part of that process is unclear? The high level process would be: 1. Build a PEX file. (Pants involved here) 2. Get it on EC2 (No Pants involved here) Step 2 really depends on how you're set up in EC2. Ypu could scp it to an instance, include it in a customoAMI, etc... I'm assuming you know how to build a PEX file using
./pants package
. If not though, have you checked out https://www.pantsbuild.org/docs/python-package-goal#creating-a-pex-file-from-a-pex_binary-target? If so and you still have questions or problems - fire away.
h

happy-kitchen-89482

05/14/2021, 1:17 AM
You could also embed the PEX file in a docker container and run that on ECS or EKS, if you're set up for that
👍 1
e

enough-analyst-54434

05/14/2021, 1:18 AM
Yeah - there are millions of ways to do step 2.
🎉 1
p

powerful-florist-1807

05/14/2021, 1:20 AM
The context is I built a PEX file for airflow dags. I can execute the dags locally, and would like to deploy the PEX to EC2.
e

enough-analyst-54434

05/14/2021, 1:20 AM
Aha. OK, so that's basically not PEX specific. You need to know how you deploy anything at all to EC2.
p

powerful-florist-1807

05/14/2021, 1:21 AM
Yea. I don't use AWS very much and need to sort out
e

enough-analyst-54434

05/14/2021, 1:23 AM
Ok - gotcha. The best advice we can probably give is PEX files are copyable. So you just need to copy them somewhere. To a docker container image that you then run on EC2 like Benjy mentioned. To an EC2 instance directly with scp. Or into an AMI image using an AMI image building tool. The only other consideration is the target for the copy has a compatible python interpreter installed and on the PATH.
p

powerful-florist-1807

05/14/2021, 1:25 AM
I think I got your idea. In my company, we typically build AMI images and deploy them to EC2. Adding PEX to the image for deploy may be the easiest solution for me which I would like to try first.
e

enough-analyst-54434

05/14/2021, 1:25 AM
As to which method to choose amongst those, or whether to use some other method ... we're probably not the best folks to ask. I can't reccomend a good cloud / AWS / EC2 forum, but I'm sure there are some out there.
h

happy-kitchen-89482

05/14/2021, 7:10 PM
Whatever method you use, the runtime basically needs 1) your .pex file and 2) a suitable python interpreter to run it on
👍 1
How exactly to get those things working really depends on how your organization wants to work with AWS
As John mentioned, we're not the best people to have an opinion about that...
p

powerful-florist-1807

05/14/2021, 7:14 PM
Sure. This is more of an internal discussion. The best way for us is to host PEX files within dockers, as you mentioned it is easier to setup the python interpreter for the docker