https://pantsbuild.org/ logo
Join Slack
Powered by
I'm evidently doing something wrong, but I have a ...
# development
h
I'm evidently doing something wrong, but I have a @rule that calls a function that raises, and that result is getting memoized. I have to kill-pantsd in order to retry the inner function.
a
hm, that's interesting. so the 
@rule
execution itself isn't getting memoized?
w
yea, Daniel was right above (or somewhere... where Henry was asking about this): 
@rules
are in general memoized.
the reason for that is that they are supposed to be deterministic
a
i was able to run `yield Get()`s in a loop in the 
--query
pipelining PR and they weren't incorrectly memoized, which was v cool
but the contents were hashed and changing so that makes sense
w
right, that's intended
@happy-kitchen-89482: what was the non-determinstic behavior in this case?
h
expensive network request
I want to memoize it, but only if it succeeds
w
@happy-kitchen-89482: rules aren't supposed to be making network requests
h
Well, something needs to...
What is the sensible way to do this?
w
h
Also, not memoizing an exception seems... normal
w
if it highlights places where we are non-deterministic, but shouldn't be...
i get the point. but there are assumptions around conventions here
discussed in those tickets
a
https://pantsbuild.slack.com/archives/C0D7TNJHL/p1572802187413900?thread_ts=1572554597.372300&cid=C0D7TNJHL was some discussion about making the engine aware of a 
RetryableRequest
object which would be usable like:
Copy code
@rule
def f() -> X:
  yield Get(Snapshot, UrlToFetch, RetryableRequest(url_to_fetch))
and would have retry parameters. but there's (generic, already-interesting) work on rule cacheability required to make that happen, discussed later in the thread.
#8455 is the cacheability ticket as stu noted
w
UrltoFetch is discussed on the first ticket: because the behavior is supposed to be side-effect free and "eventually determinstic", it can have built-in retry
☝️ 1
a
thank you, hadn't realized that
w
(i don't think it does... but it can, heh)
a
yes!
i am assuming that process execution and url fetching would be great incremental steps to a more general solution, if we need a more general solution
h
It still seems wrong, and unexpected, to memoize an exception.
Even if we solve the network issue separately.
a
^i'm pretty sure i agree with the above
if we want to cache exceptions at all i would want it to be an explicit flag, if we even allow that at all
not sure again what the immediate barriers are to doing that because i need to look into #8455 more fully
Open in Slack
PreviousNext
Powered by