https://pantsbuild.org/ logo
Join Slack
Powered by
If I understand correctly, the documentation (<htt...
# development
r
If I understand correctly, the documentation (https://github.com/pantsbuild/pants/tree/master/src/python/pants/engine#api) says that I should be able to do 
scheduler.product_request(Snapshot, [UrlToFetch])
, so long as there is an 
@rule
of the form 
@rule(Snapshot, [Select(UrlToFetch)])
. I have an intrinsic rule, introduced in https://github.com/pantsbuild/pants/pull/6660. But when I try to do this in tests it doesn’t recognize the rule:
Copy code
def test_download(self):
    scheduler = self.scheduler(rules, include_trace_on_error=True)
    url = UrlToFetch("<http://google.com>")
    snapshot, = scheduler.product_request(Snapshot, subjects=[url])
    self.assert_equal_with_printing('a', str(snapshot))
This is inside 
tests/python/pants_test/engine/test_engine.py
w
that should work, but there can be issues with Intrinsic rules
it's possible that there is no root being created for that intrinsic rule
re the dotgraph... hm! that is very intriguing
i can check out the branch and see what i can see. but certainly if that is rendered it should be possible to select it
r
Okay, the release just landed, but I will upload the changes once I have all the comms in place. Or we can pair on it if you want, whatever suits you
w
@red-balloon-89377: a workaround that should work (with a TODO on it) would be to install a little 
@rule
to act as indirection there
TestBase
can add additional rules, so if in a test you add an 
@rule
like
Copy code
class WorkaroundSnapshot(datatype([('snapshot', Snapshot)])): pass

@rule(WorkaroundSnapshot, [Select(UrlToFetch)])
def workaround(url_to_fetch):
  x = yield Get(Snapshot, UrlToFetch, url_to_fetch)
  yield x
... you can hack around this.
but yea, i'll look
r
Why not just 
@rule(Snapshot, [Select(UrlToFetch)])
? if the intrinsic rule is not recognized, this should work right?
Regardless, I was trying a similar thing just now.
w
because that would be shaped exactly the same way as the Intrinsic, and would trigger an ambiguity error
you can't have two paths through the graph that provide 
X
for 
Y
, and that's the smallest example of that
r
Oh, okay. So the problem is not that the intrinsic is not being added, but that it just can’t form a path from X to Y
w
daniel's dependent PR shows that the Intrinsic is working
i believe that the issue has more to do with using it as a 
Root
... the rule graph differentiates between roots and inner nodes ... in odd ways that i only somewhat understand and have been trying to improve
r
Okay, gotcha
Nope, still the same error: 
no rule was available to compute UrlToFetch with parameter type ()
. From the error it looks like it’s trying to build an UrlToFetch, which is weird because I give it one (this is with the tiny rule you gave me).
w
hmmm... parameter type 
()
?
what's the callsite look like now?
r
Copy code
def test_download(self):
    rules = [download]
    # rules = []
    scheduler = self.scheduler(rules, include_trace_on_error=True)
    # print("BL: scheduler.".format(scheduler._scheduler._))
    url = UrlToFetch("<http://google.com>")
    snapshot, = scheduler.product_request(WorkaroundSnapshot, subjects=[url])
    self.assert_equal_with_printing('a', str(snapshot))
w
scheduler.product_request(Snapshot, subjects=[UrlToFetch("<http://google.com>")])
should definitely identify that as a 
UrlToFetch
sorry, trying it. one sec.
hmmm, wait. i don't think that you need to create a new scheduler here
sec.
@red-balloon-89377: oh... and: what is 
download
?
(the workaround?)
sorry, sec. rewinding to your first example.
r
That is…
Copy code
@rule(WorkaroundSnapshot, [Select(UrlToFetch)])
def download(url):
  s = yield Get(Snapshot, UrlToFetch, url)
  yield
w
running it. i think the issue might be that you're creating a new scheduler there, rather than using the one on 
TestBase
. confirming
r
So in 
pants_test/engine/test_fs.py
we do create new schedulers, but we manually register all the fs rules as roots
Yep, it was that, it works in tests_fs.py
w
hmhmhmhmhmmmmmm
i would love to understand that one
i didn't think it was necessary to install anything in particular to make that work...
r
Copy code
def test_download(self):
    scheduler = self.mk_scheduler(rules=create_fs_rules())
    snapshot = scheduler.product_request(Snapshot, subjects=[UrlToFetch("<http://science-binaries.local.twitter.com/home/third_party/source/python/wheels/Babel-2.5.3-py2.py3-none-any.whl>")])
    assert(snapshot)
works
w
oooh, as roots interesting
r
(in 
test_fs.py
)
w
ahhh. ok.
so... i think the difference here is that the 
TestBase
scheduler is completely bare... with only the intrinsics
and without declaring the thing as a Root, you then can't select it
r
That would make sense, except that in my previous example, in 
test_engine.py
, the class 
EngineTest
does not inherit from 
TestBase
But yeah, I think the problem is the same
w
i think that it might actually be reasonable to require people to declare the additional rules that they want present in a unit test
so maybe this is working as intended, but the error message is bad...?
r
I agree, it might just be that I started off in the wrong file.
and if I had looked at 
test_fs.py
from the beginning this should not have been a problem
Beyond adding 
RootRule(UrlToFetch)
w
definitely interested in improving that error message though... it could happen anytime someone is attempting to do this, whether in a test or not
r
Yep, should I open a ticket?
w
sure... basically: "When no RootRule is installed for a type, the error message returned by product_request isn't clear"
thanks!
r
No problem 🙂 I will probably not have time to finish up the tests today, so feel free to pick it up/reassign, otherwise I’ll do them tomorrow.
Open in Slack
PreviousNext
Powered by