pantsbuild #development

hey gang... what's the likelihood of getting review on https://rbcommons.com/s/twitter/r/3274/ and https://rbcommons.com/s/twitter/r/3245/ ? i'm interested in making additional changes, and having outstanding reviews is a challenge

bored-art-40741

12/28/2015, 8:47 PM

I might not get much time to do OSS CRs until after the new year

bored-art-40741

12/28/2015, 8:47 PM

I have a fixit week project I'm trying to burn through and then a couple days of relaxing

witty-crayon-22786

12/28/2015, 8:50 PM

sounds good

bored-art-40741

12/28/2015, 8:51 PM

I might be able to go through them this holiday/weekend

witty-crayon-22786

12/28/2015, 8:57 PM

thanks, that would be awesome. i will be traveling, but might be able to respond in the evenings

witty-crayon-22786

12/29/2015, 7:51 PM

alright, i'm pretty confident in https://rbcommons.com/s/twitter/r/3245/ now, despite the complexity. going to add a few more tests for the product graph validation, but i think the resulting error is useful enough to ship:

witty-crayon-22786

12/29/2015, 7:52 PM

Copy code

pants.engine.exp.scheduler.PartiallyConsumedInputsError: While attempting to produce Classpath for Target(address=src/thrift/codegen/unconfigured), some products could not be consumed:
  To consume Sources(u'.thrift'):
    ScroogePlanner needed (ScroogeJavaConfiguration OR ScroogeScalaConfiguration)
    ApacheThriftPlanner needed (ApacheThriftJavaConfiguration)

witty-crayon-22786

12/29/2015, 11:00 PM

...oh, interesting. i think i just noticed that https://rbcommons.com/s/twitter/r/3274/ means that we don't need synthetic targets for codegen...?

witty-crayon-22786

12/29/2015, 11:01 PM

will explore that a bit today.

witty-crayon-22786

12/30/2015, 1:51 AM

hm... as far as i can tell, we still need synthetic targets or some equivalent. in order to kill them we'd have to make Configuration/Sources into products which held their dependencies. but then you'd need the ability to partially execute the graph to generate a Sources product, and then request more promises for dependencies

witty-crayon-22786

12/30/2015, 2:01 AM

i do think that

dependencies

should be members of Sources/Configuration rather than a Target though, so will do that as part of 3274

happy-kitchen-89482

12/30/2015, 4:02 PM

I think we need synthetic configuration, not synthetic targets?

happy-kitchen-89482

12/30/2015, 4:03 PM

pat.rick put a bunch of thought into this

witty-crayon-22786

12/30/2015, 5:29 PM

yes: synthetic configuration

witty-crayon-22786

12/30/2015, 5:29 PM

ie, products

witty-crayon-22786

12/30/2015, 5:30 PM

but to make it truly useful, you need to support including new dependencies at that point

witty-crayon-22786

12/30/2015, 6:51 PM

re-reading the skyframe docs has been enlightening now that i have a better understanding of the problem space; highly recommended. http://bazel.io/docs/skyframe.html

witty-crayon-22786

12/30/2015, 6:52 PM

in particular, it doesn't separate "planning" from "execution"

witty-crayon-22786

12/30/2015, 6:53 PM

cc @bored-art-40741 : very interested in your thoughts on that matter.

bored-art-40741

12/30/2015, 6:53 PM

I'll take a look

bored-art-40741

12/30/2015, 6:54 PM

Now that I did a 180 on caching semantics, it might be worth reconsidering the separation of planning and execution

witty-crayon-22786

12/30/2015, 6:54 PM

the end result seems to be that the execution graph is not complete until it finishes executing

bored-art-40741

12/30/2015, 6:54 PM

Since we've hit at least one case where planning is obviously cyclical without seeing how the execution short circuits