https://pantsbuild.org/ logo
Join Slack
Powered by
if i wanted to integrate boto3 with a plugin, is t...
# plugins
f
if i wanted to integrate boto3 with a plugin, is there a correct way to do that that fits into the plugin execution model? it doesn't seem like multithreaded blocking APIs fit will into the curent rule API
h
I haven't worked a ton with boto. How are you hoping to use it? My first instinct is that you could run a single 
InteractiveProcess
in the foreground by executing a Pex with boto3 loaded. I'm not sure that matches what you're hoping for, though
f
boto3 is the AWS multitool, i'd use it for sending AWS API commands; it's normally used as a python lib
building a pex and running it as a process doesn't sound like it matches the use cases i have in mind
is there a way write a rule that runs in another thread? that's how i would approach this with a "normal" python async application if there were no lib to replace this
h
I don't think a very tight way to force it to run in another thread. Iiuc, we use Rust's Tokio crate to coordinate launching threads, i.e. the scheduler @witty-crayon-22786 thoughts on this?
w
is this a read only API? reminder that 
@rules
should not have sideeffects
➕ 1
spawning a thread associated with a Subsystem or something might be ok. but since rules don’t have a supported way to mutate objects, holding onto a reference to the thread, or waiting for it to complete would be challenging
if the usage of the API is sideeffecting, spawning an InteractiveProcess in the foreground in a @goal_rule would be the recommended way.
@goal_rules
are the only thing that are supposed to be able to “change the world”
f
a deployment to a cloud environment is definitely a "change the world" kinda thing; i wasn't aware that all @rule functions are supposed to be side-effect free
my docker plugin is definitely not side-effect free, as running 
docker build
is done purely for side-effects that are not visible to anything in the pants process execution space
w
Side effect free or idempotent, really. 
docker build
might be idempotent
Deploy steps definitely qualify as the kind of stuff that should be in a 
@goal_rule
👍 1
I am currently working on a blog post that explains some of the motivation here.
👍🏻 1
f
okay this makes sense, and clarifies why you wouldn't want to integrate deployment tooling so directly as a plugin
perhaps something like building a deployment app as a pex and then calling it as an 
InteractiveProcess
makes more sense then
👍 2
h
That's how we've been envisioning implementing a future publish goal. Creating the artifacts is parallelizable and cacheable, deploying is not
f
no and after thinking through it, it makes more sense, as for many teams, tools for deployment are often provided outside the team; a better strategy for dealing with that would be rules that can set up execution environments for tools to run in
👍 1
w
yea, that sounds exactly right.
2 Views
Open in Slack
PreviousNext
Powered by