https://pantsbuild.org/ logo
#development
Title
# development
h

happy-kitchen-89482

11/12/2019, 6:22 PM
I'm evidently doing something wrong, but I have a @rule that calls a function that raises, and that result is getting memoized. I have to kill-pantsd in order to retry the inner function.
a

aloof-angle-91616

11/12/2019, 6:23 PM
hm, that's interesting. so the
@rule
execution itself isn't getting memoized?
w

witty-crayon-22786

11/12/2019, 6:24 PM
yea, Daniel was right above (or somewhere... where Henry was asking about this):
@rules
are in general memoized.
the reason for that is that they are supposed to be deterministic
a

aloof-angle-91616

11/12/2019, 6:25 PM
i was able to run `yield Get()`s in a loop in the
--query
pipelining PR and they weren't incorrectly memoized, which was v cool
but the contents were hashed and changing so that makes sense
w

witty-crayon-22786

11/12/2019, 6:25 PM
right, that's intended
@happy-kitchen-89482: what was the non-determinstic behavior in this case?
h

happy-kitchen-89482

11/12/2019, 6:29 PM
expensive network request
I want to memoize it, but only if it succeeds
w

witty-crayon-22786

11/12/2019, 6:29 PM
@happy-kitchen-89482: rules aren't supposed to be making network requests
h

happy-kitchen-89482

11/12/2019, 6:30 PM
Well, something needs to...
What is the sensible way to do this?
w

witty-crayon-22786

11/12/2019, 6:30 PM
h

happy-kitchen-89482

11/12/2019, 6:30 PM
Also, not memoizing an exception seems... normal
w

witty-crayon-22786

11/12/2019, 6:32 PM
if it highlights places where we are non-deterministic, but shouldn't be...
i get the point. but there are assumptions around conventions here
discussed in those tickets
a

aloof-angle-91616

11/12/2019, 6:33 PM
https://pantsbuild.slack.com/archives/C0D7TNJHL/p1572802187413900?thread_ts=1572554597.372300&cid=C0D7TNJHL was some discussion about making the engine aware of a
RetryableRequest
object which would be usable like:
Copy code
@rule
def f() -> X:
  yield Get(Snapshot, UrlToFetch, RetryableRequest(url_to_fetch))
and would have retry parameters. but there's (generic, already-interesting) work on rule cacheability required to make that happen, discussed later in the thread.
#8455 is the cacheability ticket as stu noted
w

witty-crayon-22786

11/12/2019, 6:35 PM
UrltoFetch is discussed on the first ticket: because the behavior is supposed to be side-effect free and "eventually determinstic", it can have built-in retry
☝️ 1
a

aloof-angle-91616

11/12/2019, 6:39 PM
thank you, hadn't realized that
w

witty-crayon-22786

11/12/2019, 6:40 PM
(i don't think it does... but it can, heh)
a

aloof-angle-91616

11/12/2019, 6:40 PM
yes!
i am assuming that process execution and url fetching would be great incremental steps to a more general solution, if we need a more general solution
h

happy-kitchen-89482

11/12/2019, 6:57 PM
It still seems wrong, and unexpected, to memoize an exception.
Even if we solve the network issue separately.
a

aloof-angle-91616

11/12/2019, 7:56 PM
^i'm pretty sure i agree with the above
if we want to cache exceptions at all i would want it to be an explicit flag, if we even allow that at all
not sure again what the immediate barriers are to doing that because i need to look into #8455 more fully