New :thread: for immutable inputs symlinking sourc...
# development
b
New ๐Ÿงต for immutable inputs symlinking source dirs
Naturally, the code is going to symlink in-repo source code if it exceeds the file threshold (but not for formatter/fixer process invocations) This is certainly sub-optimal as the same exact subset of the sources likely won't be used across sandboxes. But c'est la vie. The real "issue" though is if users expect (or more accurately should be relying on) their sandbox to be mutable. Namely should they be able to add files to directories?
Although it might be surprisng to some, I lean towards "no you shouldn't assume its writable" After all the purpose of these sandboxes are hermicity and reproducibility. Conceivably, those could be poisoned if the running code starts adding/modifying files
f
I agree with that assumption especially since if a
Process
modifies its input root, that modification will only be visible outside of the execution sandbox (whether local or in remote execution) if it is captured as an explicit output path (
output_files
/
output_directories
). By default, modifications (even new files) are not visible outside of the execution.
๐Ÿ™Œ 1
โœ… 1
Furthermore, very reasonable to require that the input root won't be modified to allow for the symlink optimization. (And maybe have a flag to
Process
that can disable the symlink optimization if there are ever outlier
Process
invocations that will violate the assumption.)
Plus it would support
node_modules
and Ruby bundler gem install directories to always symlink those directories.
b
And maybe have a flag to
Process
that can disable the symlink optimization
Yeah that's planned for after my PR ๐Ÿ™‚
f
And engine could even monitor those file paths like it does for
PathGlobs
usage to invalidate any
Process
that relied on a symlink.
Tangentially, should the API for
Process
provide for a "always symlink this source dir" attribute? (Versus just doing a heuristic.)
b
Thats also coming too ๐Ÿ˜‚
We'll have both inlcude and exclude lists for maximum custopmizability plus the huerstic for when we (the user) don't control the process
f
Cool, great to see this work happening!
๐Ÿš€ 2
b
๐Ÿ™‚