#development
Title
h
happy-kitchen-89482
08/09/2019, 5:36 PMIt would be really nice to write code that's pants-agnostic, but whose datatypes can be used as Pants products. So rules can be relatively thin wrappers around functionality that knows nothing of pants.
👌 2
a
aloof-angle-91616
08/09/2019, 5:37 PMyes!!!!
w
witty-crayon-22786
08/09/2019, 5:37 PMthat's fairly unlikely due to the need for sandboxing
a
aloof-angle-91616
08/09/2019, 5:37 PMstu and i so far disagree on this point
w
witty-crayon-22786
08/09/2019, 5:38 PMwe're not enforcing it as well as we need to be in the long term
examples: 1) no network access, 2) no filesystem access, 3) etc
4) no sideeffects
a
aloof-angle-91616
08/09/2019, 5:39 PMi see that as something we can arrive at, along with developing python tools as benjy depicts that can work in that environment instead of accessing the filesystem directly
w
witty-crayon-22786
08/09/2019, 5:39 PMwell, sure. but that means the opposite of "knows nothing of pants"
h
happy-kitchen-89482
08/09/2019, 5:40 PMWell, rules today can call arbitrary code that can side-effect etc.
We're relying on discipline, no?
w
witty-crayon-22786
08/09/2019, 5:40 PMright: as i said, we're definitely not doing well at this yet
a
aloof-angle-91616
08/09/2019, 5:40 PMi view "tools that can be used in a sandboxed model" as a strict superset of tools that can't
w
witty-crayon-22786
08/09/2019, 5:40 PMsubset?
a
aloof-angle-91616
08/09/2019, 5:40 PMsuperset!
h
happy-kitchen-89482
08/09/2019, 5:41 PMbut any enforcement would presumably have to introspect the rule and all its dependencies?
So those dependencies don't have to "know about pants", although they do need to not have side effects...
?
a
aloof-angle-91616
08/09/2019, 5:42 PMwhile i usually just think of "enforcement" as "hopefully we can make everything that doesn't assume a sandboxed environment break early and loudly"
w
witty-crayon-22786
08/09/2019, 5:43 PM@happy-kitchen-89482: no. generally sandboxes use hooks or full replacements of API calls to make them fail
and the effect of that is "the thing fails", and you need to change your usage
h
happy-kitchen-89482
08/09/2019, 5:44 PMBut presumably those are monkeypatched
The calling code thinks its calling the normal stdlib functions?
So it needs to know that there are stdlib functions it must not call, sure, but it doesn't need to depend on Pants, is more my point.
If I'm understanding this sandbox implementation correctly.
w
witty-crayon-22786
08/09/2019, 5:46 PMRight. So it must not do filesystem access via read/open. But then it needs to know how to do it correctly
(or just be less useful)
a
aloof-angle-91616
08/09/2019, 5:48 PMit's very possible to pass e.g. file handles through the engine and i am using that right now to implement a v2 ruleset which communicates via BSP to a subprocess. that wouldn't lead to a remotable process execution, but it does work with e.g.
json.dump()w
witty-crayon-22786
08/09/2019, 6:03 PM@aloof-angle-91616: er, but that is not code that we would ship. it totally violates the sandbox
like, we'd have to make it part of the API
a
aloof-angle-91616
08/09/2019, 6:03 PMit is an extension of https://github.com/pantsbuild/pants/issues/7079
so yes
w
witty-crayon-22786
08/09/2019, 6:04 PMif we actually build it in as an intrinsic, so it's a supported API of the sandbox, then fine. but that's definitely not "unmodified python code"
a
aloof-angle-91616
08/09/2019, 6:04 PMthat is exactly what i am thinking
the process currently runs one-shot to completion within a v1 pants task
i want to figure out restarting a v2 rule (e.g. one that processes messages from a subprocess) in order to introduce this as an actually performant solution
but have been avoiding that for now
h
happy-kitchen-89482
08/09/2019, 6:47 PMI guess the specific example I have in mind doesn't require filesystem access or process execution, so that's coloring my thinking...
w
witty-crayon-22786
08/09/2019, 7:20 PMyea
a
aloof-angle-91616
08/09/2019, 7:22 PMit's actually pretty easy to write python that doesn't do this, imho. not sure if anyone agrees with me yet
w
witty-crayon-22786
08/09/2019, 7:23 PMthe example you used talked about passing in a socket =/
i'm sure there are cases, i just don't know how widespread they are, and my assumption is: not very
a
aloof-angle-91616
08/09/2019, 7:37 PMyes, and i think passing in a socket can be made cacheable
w
witty-crayon-22786
08/09/2019, 7:39 PMyes. but it would be exceedingly unlikely to be unmodified python
a
aloof-angle-91616
08/09/2019, 7:39 PMworking on it!!