it might be an optimization though when spawning many, many processes on a local machine to have a set of “canonical” materialized files (for commonly-materialized files) and then use symlinks in /tmp to those canonical files
i’m gonna maybe play with this on the first jump to dallas from el paso
12/30/2019, 11:25 PM
some tools will care whether something is a symlink or not, so would need to be optional on a tool-by-tool basis. but maybe safe enough to do by default.
12/31/2019, 1:43 AM
if this seems like it’s doable without broad architectural changes (i think so!) i will try to put up a draft PR for discussion