https://pantsbuild.org/ logo
Join Slack
Powered by
I only see examples for using `Process` (or a subc...
# general
h
I only see examples for using 
Process
(or a subclass of it). What could I reference for when I don't need this functionality (e.g. the library doing the work is a python library in the plugin environment and not a call out to the system)?
On the plus side, this seems to be the last hurdle. I've gotten as far as verifying that 
pants.toml
can contain my new repositories options and that 
pex_binary
shows up with a 
./pants list publish
w
How do you mean that "the library doing the work is a python library in the plugin environment and not a call out to the system"
In most of the plugins, a Process is created containing the pertinent files, environment, executables, args, etc
h
So I have 
rules.py
(much like your blog). I have a third party module that can do the things I need. I don't need to call out to some binary on the system.
I just need to run a couple methods using the tooling in https://github.com/devopshq/artifactory
In most of the plugins, a Process is created
Yeah, that's kind of my issue. I don't need to follow that pattern 😞
w
Really? But like... You need to call something to execute on your files, right?
Note: Not trying to be difficult, just trying to understand
h
The call looks like
Copy code
import artifactory

artifactory.<my call here>
w
Ah, okay, so the call is performed in Python in your plugin rule directly
h
Yeah, that's what I was hoping to do. Just not sure on the proper semantics for that with how much asynchronous stuff there is.
Trying to learn how the 
pytest
stuff works internally cause maybe that's similar
w
Well, I mean, I don't think you need the Process per se - just running your code in your rules against the files, isn't that enough? As in, if you have the ability to call your rule, whatever is in there is up to you, I think.
Personally, I would probably make a 
PexProcess
out of it, but 🤷
I've been a bit out of the loop, but is this for a 
package
goal?
h
Trying to extend 
publish
for a 
pex_binary
There's a couple things that concern me about making the call in my 
@rule
function. Namely, I've seen some mention about reading from disk causing issues with caching. I would have to get credentials from a location on disk for this goal to work.
But when I was looking at the docker implementation, it seemed like publish stuff generally isn't cached and it just runs every time so maybe that's not an issue.
h
You'll only want to be careful that 
artifactory
is not doing side-effecty things like reading from the file system. If it is, Pants won't know how to cache it properly
w
Yeah, that's what I was also worried about.
h
Yeah, but for publishing, I don't think actions are cached anyway. e.g. the docker stuff has 
cache_scope=ProcessCacheScope.PER_SESSION
which I followed as "not caching"
h
if `artifactory`'s stuff is slow, it can also be worth wrapping into a 
Process
so that things get cached to disk. Otherwise, it only gets memoized to memory w/ pantsd
worth wrapping into a 
Process
By building a PEX w/ 
artifactory
, then invoking it in a 
Process
Yeah, but for publishing, I don't think actions are cached anyway.
That's the issue - Pants will think it's safe to memoize and won't invalidate for you
There is a way to say a rule is not cachable, I think 
@_uncacheable_rule
iirc
h
interesting. Unfortunately, I feel like I'm unaware of the ecosystem and capabilities so that I don't know what the right way to structure this is.
I think trying to understand how pytext works is my best bet so that I can make a separate pex for what the artifactory library needs to do and then invoke that as a 
Process
w
It looks a bit odd, because I think Artifactory kinda emulates Pathlib or something in how it operates. I can see why this is hard to model
Does each command in the Artifactory lib run at interpreter time? Or, does the library bundle the commands up and replay them?
h
They run at interpreter time. And yes, the decision to be an extension of pathlib is an interesting choice.
The company has a cli that runs on the system that I could use, but I'm trying to avoid asking all our developers to set up a system package.
But actually, I might go down that path since they have an 
npm
install option
Thanks a bunch for your help, btw. It's definitely hard to articulate what I'm trying to do when there's so much unknown. Appreciate the patience very much.
🙌 1
3 Views
Open in Slack
PreviousNext
Powered by