https://pantsbuild.org/ logo
Join Slack
Powered by
It seems process output capturing-and-re-materiali...
# plugins
b
It seems process output capturing-and-re-materialization is turning symlinks into copies. Is that expected? Any way to preserve the symlink?
f
The Rust code that materializes and captures sandboxes would need to be updated to understand symlinks. There is already support in the Remote Execution API ("REAPI") protobuf structs for symlinks; the materialize/capture logic needs to use it.
I started doing this in https://github.com/tdyas/pants/tree/reapi_symlink_support but haven't had time to work on it. And am unlikely to have any time to work on it.
b
Ah thats a huge bummer 😕
f
Any interest in taking over the PR?
b
pooooooosibly. I have a decent Pants backlog, but this is technically work-driven so I have a bit more of a budget of time to spend. SWAG on time estimate?
w
(we had two tickets, so i closed one of them)
f
I'd like to say no more than a week.
And I'm available for a video/audio call for some knowledge transfer and/or pair programming.
I think the bulk of the time would probably be orienting yourself to the Rust code that does materialize/capture.
b
I'm good at cargo culting 🙂
f
and the gist of the change is adding a "symlink" variant to the 
Entry
enum (which my branch already does) and then add support in the materialize and capture code paths for that variant (alongside the 
File
and 
Directory
variants)
conceptually it isn't a "big" change
b
Yeah seems conceptually straightforward
f
I put the branch aside when I was trying to figure out the code that checks for duplicate symlinks in a Digest
just because it exceeded the time box I gave it for a non-priority project
but that code can be cargo culted from the duplicates checking code for files
b
Ah to check for a collision when merging?
f
yes
b
Makes sense
f
and looking at the diff, I may have already implemented the merge error detection
any ways, if interested in taking over the PR, let me know! 🙂
b
I'm gonna splash at it for a bit while I'm in monday meeting marathons
Yeah the collisions thing is getting me, lol 😵
ah got it 🙂
👍 1
❤️ 1
Well that was the easy part. I miiiiiight need some help on 
fs/store/src/lib.rs
changes
f
feel free to ask away here. and pair programming offer remains open if you want it.
b
Might take you up on that, got a calendar tomorrow I can slice?
Currently looking at: • Subsetting a Snapshot • 
contents_for_directory
• 
ensure_local_has_recursive_directory
• and maybe 
exists_recursive
i really don't know
f
any time after 3pm Eastern time tomorrow
b
Is there some reading I can do on `Digest`/`DigestTrie`/`store`?
f
I recall that the PR that introduces 
DigestTrie
might have the “theory of operation” or link to the design.
b
Today is wrought with drs appts 😞 Let me find that PR. I might also get PR running and we can discuss there too
I opened a WIP PR. It compiles, next step is testing 🙂 There's some `todo!`s in the remote section https://github.com/pantsbuild/pants/pull/16844
Thanks for the first pass y'all.
I'm having trouble finding the best way to test this outside of 
fs_test.py
, which requires path_stats to be implemented. 😞
So I guess I'm gonna implement that. Really sorry for the PR bloat 😞
Open in Slack
PreviousNext
Powered by