https://pantsbuild.org/ logo
#development
Title
# development
a

aloof-angle-91616

03/05/2020, 6:08 PM
tried to capture a moonshot that i think could effectively supersede the need for buildozer in pants, based on a conversation in #general with @average-vr-56795 many, many moons ago: https://docs.google.com/document/d/1P73tV8qOUtdqWzJtNfDmcUFLw8gtnJrFVFDR5WuxTL8/edit?usp=sharing. not sure where the exact right convergence of ideas is here but i think at twitter that we've separately identified the need for "a description of the pants graph that external cli tools can consume", as well as "buildozer that works on pants BUILD file macros, and supports set literals". i'm dumping this here because i don't have time to think more about it right now, but i plan to investigate this further this weekend.
h

hundreds-father-404

03/05/2020, 6:11 PM
The intersection with V1 BuildFileManipulator seems relevant. That’s the basis for Toolchain’s buildgen Are you proposing something like extending BuildFileManipulator or creating something new?
a

aloof-angle-91616

03/05/2020, 6:13 PM
i have absolutely no clue what BuildFileManipulator is but i was exactly thinking that one of the use cases for this would be to define templatized buildgen workflows, so i think we are probably aligned in spirit
h

hundreds-father-404

03/05/2020, 6:13 PM
Its part of the buildgen contrib package, which I believe was added by Foursquare. BuildFileManipulator understands how to update actual BUILD file entries
a

aloof-angle-91616

03/05/2020, 6:14 PM
ok, that sounds very interesting
💯 1
looking now
thank you!!!
❤️ 1
BuildFileManipulator is really useful prior art and i'm going to try to avoid losing any of the wisdom contained in its comments
stu or benjy was talking about JSON BUILD files a while ago and i was thinking that a solution i'd like to see that might actually stand up to buildozer is something that would be able to symbolically manipulate a structured representation
and i definitely see that growing out of the BuildFileManipulator code, but i would like to see specifically whether we can make the equivalent of buildozer scripts, but expressed directly in real python
still vague thoughts atm
i found instagram's libCST (https://libcst.readthedocs.io/en/latest/codemods_tutorial.html) which makes it really, really nice to rewrite python, and it seems incredibly well-designed, and i know facebook actually relies on this -- they make some absolutely fantastic program analysis tools over there. my first thought about attacking this is to try to make an analogue for every buildozer command in https://github.com/bazelbuild/buildtools/blob/master/buildozer/README.md, which seems like an incredibly fun friday night to me.
one thing that i wanted to improve on over the existing BuildFileManipulator was to avoid having to worry about modifying whitespace. libCST handles all of that, and also has truly fantastic parsing error messages & context: https://libcst.readthedocs.io/en/latest/parser.html#syntax-errors. i really hope that the API we expose over libCST can be as concise as buildozer's CLI -- but expressible in python code! which i think makes it much more reusable. i think that starlark is on the syntax level a subset of python 2, so if this approach to BUILD file rewriting is as nice as i think it is, we could even consider implementing buildozer in this way. unfortunately i currently suspect buildozer maintainers would bristle at using a python-specific API, as they've made it clear they (very wisely) don't want to be constrained to being a pure python subset syntax-wise.
also, very helpfully, this can be a v2
@goal_rule
, so we can delegate e.g. to
--query
to decouple target selection from target manipulation
but if this paradigm works nicely, i would still like to propose the idea to buildozer to use this "rewrites as code" approach. i am hopeful that it will be very fun and useful