https://pantsbuild.org/ logo
#general
Title
# general
h

high-yak-85899

05/19/2022, 11:47 PM
I only see examples for using
Process
(or a subclass of it). What could I reference for when I don't need this functionality (e.g. the library doing the work is a python library in the plugin environment and not a call out to the system)?
On the plus side, this seems to be the last hurdle. I've gotten as far as verifying that
pants.toml
can contain my new repositories options and that
pex_binary
shows up with a
./pants list publish
w

wide-midnight-78598

05/20/2022, 12:52 AM
How do you mean that "the library doing the work is a python library in the plugin environment and not a call out to the system"
In most of the plugins, a Process is created containing the pertinent files, environment, executables, args, etc
h

high-yak-85899

05/20/2022, 12:53 AM
So I have
rules.py
(much like your blog). I have a third party module that can do the things I need. I don't need to call out to some binary on the system.
I just need to run a couple methods using the tooling in https://github.com/devopshq/artifactory
In most of the plugins, a Process is created
Yeah, that's kind of my issue. I don't need to follow that pattern 😞
w

wide-midnight-78598

05/20/2022, 12:56 AM
Really? But like... You need to call something to execute on your files, right?
Note: Not trying to be difficult, just trying to understand
h

high-yak-85899

05/20/2022, 12:56 AM
The call looks like
Copy code
import artifactory

artifactory.<my call here>
w

wide-midnight-78598

05/20/2022, 12:57 AM
Ah, okay, so the call is performed in Python in your plugin rule directly
h

high-yak-85899

05/20/2022, 12:58 AM
Yeah, that's what I was hoping to do. Just not sure on the proper semantics for that with how much asynchronous stuff there is.
Trying to learn how the
pytest
stuff works internally cause maybe that's similar
w

wide-midnight-78598

05/20/2022, 12:59 AM
Well, I mean, I don't think you need the Process per se - just running your code in your rules against the files, isn't that enough? As in, if you have the ability to call your rule, whatever is in there is up to you, I think.
Personally, I would probably make a
PexProcess
out of it, but 🤷
I've been a bit out of the loop, but is this for a
package
goal?
h

high-yak-85899

05/20/2022, 1:01 AM
Trying to extend
publish
for a
pex_binary
There's a couple things that concern me about making the call in my
@rule
function. Namely, I've seen some mention about reading from disk causing issues with caching. I would have to get credentials from a location on disk for this goal to work.
But when I was looking at the docker implementation, it seemed like publish stuff generally isn't cached and it just runs every time so maybe that's not an issue.
h

hundreds-father-404

05/20/2022, 1:03 AM
You'll only want to be careful that
artifactory
is not doing side-effecty things like reading from the file system. If it is, Pants won't know how to cache it properly
w

wide-midnight-78598

05/20/2022, 1:04 AM
Yeah, that's what I was also worried about.
h

high-yak-85899

05/20/2022, 1:05 AM
Yeah, but for publishing, I don't think actions are cached anyway. e.g. the docker stuff has
cache_scope=ProcessCacheScope.PER_SESSION
which I followed as "not caching"
h

hundreds-father-404

05/20/2022, 1:05 AM
if `artifactory`'s stuff is slow, it can also be worth wrapping into a
Process
so that things get cached to disk. Otherwise, it only gets memoized to memory w/ pantsd
worth wrapping into a
Process
By building a PEX w/
artifactory
, then invoking it in a
Process
Yeah, but for publishing, I don't think actions are cached anyway.
That's the issue - Pants will think it's safe to memoize and won't invalidate for you
There is a way to say a rule is not cachable, I think
@_uncacheable_rule
iirc
h

high-yak-85899

05/20/2022, 1:07 AM
interesting. Unfortunately, I feel like I'm unaware of the ecosystem and capabilities so that I don't know what the right way to structure this is.
I think trying to understand how pytext works is my best bet so that I can make a separate pex for what the artifactory library needs to do and then invoke that as a
Process
w

wide-midnight-78598

05/20/2022, 1:10 AM
It looks a bit odd, because I think Artifactory kinda emulates Pathlib or something in how it operates. I can see why this is hard to model
Does each command in the Artifactory lib run at interpreter time? Or, does the library bundle the commands up and replay them?
h

high-yak-85899

05/20/2022, 1:58 AM
They run at interpreter time. And yes, the decision to be an extension of pathlib is an interesting choice.
The company has a cli that runs on the system that I could use, but I'm trying to avoid asking all our developers to set up a system package.
But actually, I might go down that path since they have an
npm
install option
Thanks a bunch for your help, btw. It's definitely hard to articulate what I'm trying to do when there's so much unknown. Appreciate the patience very much.
🙌 1
3 Views