https://pantsbuild.org/ logo
#development
Title
# development
b

bitter-ability-32190

09/12/2022, 5:11 PM
For my own understanding, it kinda seems like most Pants subprocess should be using
immutable_input_digests
as most won't be mutating inputs (save for
fmt
?) Is that true? Should we consider swapping the "default" so we have
input_digests
be immutable and
mutable_input_digest
for the alternative?
1
I suppose some tools aren't very symlink friendly. So there's them
f

fast-nail-55400

09/12/2022, 5:13 PM
"immutable" really means that the
immutable_input_digest
is something to be reused over multiple
Process
invocations
like SDKs etc. that are invariant over those
Process
invocations whereas the source files will change for each
Process
invocation
b

bitter-ability-32190

09/12/2022, 5:14 PM
Oh hmmm, so immutable over time, not immutable in a sandbox?
f

fast-nail-55400

09/12/2022, 5:14 PM
right
you make a good point though about source files not changing in a sandbox
bazel creates symlinks from the sandbox back to the source tree in recognition of that
b

bitter-ability-32190

09/12/2022, 5:15 PM
Yeah we have some large resources I'd love to "materialize" much faster 🙂
f

fast-nail-55400

09/12/2022, 5:16 PM
then might be an idea to think about for Pants but it would still be a separate idea from
immutable_input_digest
which is geared to things like unpacking the Go SDK once and then re-using it
b

bitter-ability-32190

09/12/2022, 5:16 PM
Makes sense
f

fast-nail-55400

09/12/2022, 5:17 PM
(although re Go SDK, we used to merge it in as an input digest and switched to requiring it to be installed locally because immutable_input_digest support had not been implemented yet)
b

bitter-ability-32190

09/12/2022, 5:17 PM
I also want this feature for
GeneratedSources
I have resource dependencies which are generated and are immutable over time (versioned sys libraries)
h

hundreds-father-404

09/12/2022, 5:19 PM
We indeed should likely be using immutable_input_digests in more places than we are now, e.g. https://github.com/pantsbuild/pants/issues/14070 Stu has only proposed that we should simplify the API before we port a bunch of stuff
1
b

bitter-ability-32190

09/12/2022, 5:20 PM
I'm happy to be a sounding board/extra brain on the topic.
🧠 2
w

witty-crayon-22786

09/12/2022, 5:25 PM
at one extreme, we could take the idea from https://github.com/pantsbuild/pants/issues/14070 (includelist paths that should be immutable) and invert it to excludelisting paths which should be mutable
👀 1
w

wide-midnight-78598

09/12/2022, 5:25 PM
Not that it would speed anything up, but is there already the ability to create a sandbox with
ro
properties on files?
w

witty-crayon-22786

09/12/2022, 5:26 PM
but the current implementation would leave a lot of garbage behind… there is some amount of a hint around re-use inherent in marking something
immutable
currently, because we keep it on disk for the lifetime of
pantsd
@wide-midnight-78598: somewhat: immutable_inputs are marked readonly, since they will be shared between processes.
w

wide-midnight-78598

09/12/2022, 5:27 PM
I just started thinking about 3rd party deps tools being good citizens in the sandbox, and basically enforcing that with attrs, and maybe even users
Kinda tangential to this convo though - so I'll just mindlessly ponder
w

witty-crayon-22786

09/12/2022, 10:58 PM
mmm, so! there is another potential change here as of late. the
brfs
filesystem hosts digests as a FUSE filesystem. we discussed using it as an alternative to implementing something like immutable inputs, but at the time, FUSE was far too unstable on macOS to be a solution. in the last week or so though, https://www.fuse-t.org/ was announced, which replaces the unstable portion on macOS, and apparently improves performance
2
oh. but i had forgotten about this comment: https://github.com/pantsbuild/pants/issues/12716#issuecomment-982131890 … that’s probably still true? i.e., we would basically want to implement the
immutable_inputs
in terms of FUSE, rather than necessarily having the entire sandbox be FUSE.