Hi Everyone, I'm just getting started with pants, ...
# general
l
Hi Everyone, I'm just getting started with pants, trying to move my team to a monorepo. I've been experimenting with setting up a test project to get a feel for pants, but my updates to files are not getting picked up. For example, my source root pattern is "src/*". if I run pants roots, I see 3 directories I made, A, B, and C. If I add directory D and rerun pants roots, I still only see A, B, and C. I noticed that if I make an adjustment to my pants.toml file, it reinitializes the scheduler and the changes are then picked up. After making an adjustment and rerunning pants roots, I now see A, B, C and D. This is just an example with roots, but the same issue is happening with any changes I make to BUILD files, which is incredibly frustrating. Has anyone else experienced this? I'm currently on Windows 10 and using WSL
👋 2
w
That seems very strange, I personally haven't run into this, but maybe someone else has. In the meantime, is this something you could create a github sample for, or share your pants.toml/other code? I could see if I can repro on my machine
l
w
I just tried on my Mac - and
roots
was picking up the new folders on every command, without any funny business, I'll grab your repo and try now
l
Thank you for checking this out with me!
w
Copy code
sj@tinyrick pants_exploration % ./pants roots  
src/lib
src/projects
src/tools

sj@tinyrick pants_exploration % mkdir src/helloworld

sj@tinyrick pants_exploration % ./pants roots       
src/helloworld
src/lib
src/projects
src/tools
Seems to work fine - I wonder if this is a WSL2 thing? Or, if maybe there is some strange
AllRoots
caching, but only on windows?
l
Must be WSL2, it's not just the roots, but any changes I make, including BUILD files
I'll continue to search around, I appreciate the help!
w
🤯
Would you mind trying something out?
./pants -ldebug roots
mkdir src/123456
./pants -ldebug roots
Curious if the debug logs shows anything, both "before" and "after" a change
One other idea I'd be curious about... In your pants.toml, maybe try stopping the pants daemon from running and see if that makes a difference:
Copy code
[GLOBAL]
pantsd = false
l
aha, turning off pantsd worked
w
Hmm, that's both good and unfortunate. This was created yesterday, is this yours? https://github.com/pantsbuild/pants/issues/16534
l
Not me, but that's the exact issue I am having.
w
This was also referenced: https://github.com/pantsbuild/pants/issues/10445 I'll make an update to that issue - but maybe see if the home directory workaround works with the pants daemon on?
l
sorry, what do you mean by home directory workaround
w
https://github.com/pantsbuild/pants/issues/10445 This one: "Moving the repository in the users home directory (
~
) fixes the issue."
That was pants 1, but worth seeing if the behaviour is similar
l
Yes, moving to home and re-enabling pantsd worked
w
🤯 🤯 🤯 🤯
Unfortunately, as I don't know how the daemon works, nor have I used WSL in quite some time, I'm not much more help. My guess is around permissions, but 🤷
e
@late-ram-12573 what does
df -h
say? IIUC WSL2 uses, of all things, 9p for mounting Windows drives / folders. That's a network filesystem (from Plan 9!!) and network filesystems pretty famously don't work with many more advanced fs operations like locking and event notifications in general (NFS v4 I think finally mostly does work with locking, not sure about fs events).
l
Sorry, not sure what to look for here:
w
In reading more about WSL2, it seems a lot of people are having problems of the "file changed" variety due to inotify events not being sent up, or other reasons. Web dev seems hit hard in particular
e
@late-ram-12573 it's the 1st filesystem column which Windows unhelpfully productizes and just says C:\ - which is not a filesystem 🤦
But, pretty sure it's 9p and that's the issue here.
Ah, my bad actually. How about
df -T
?
That should give a type column.
l
image.png
using /mnt/d for my project, so it is 9p
e
Aha, ok. That is definitely it then. Pants really doesn't work with network filesystems.
I think everyone who uses Pants regularly can agreed
--no-pantsd
is not really any sort of solution at all. FS event handling is critical to perf.
l
That's unfortunate, but I certainly appreciate the help.
e
It is. Really we get lucky WSL exists at all since we've never expended the effort to get Pants and its backends all working specifically on Windows. All the developers back through 2010 inception have been on Linux or MacOS and so this support has never come.
I know there has been recent mainlining of the Paragon NTFS driver into the Linux kernel. I wonder if your kernel is new enough / can be upgraded to one where you can just re-mount as ntfs?
@late-ram-12573 what does
cat /proc/filesystems
say? Any ntfs in there?
l
just nfs
image.png
e
Yeah - ok.
Have you tried
apt update && apt install ntfs-3g
?I think WSL(2) defaults to an Ubuntu / Debian image.
Ah, sorry - wrong package. That's the Fuse-based impl. May work, but should be slow.
l
I can try that but I do have a question for you. In the issues https://github.com/pantsbuild/pants/issues/16534 the opener mentioned deleting the .pids folder before running pants commands worked. I just tried it and it does for me as well. It's a "hacky" solution, but I added a command to delete the .pids folder when running the pants shell script, and it seems to work. Is this a terrible idea?
h
I think that might cause pantsd to restart on every run… ? So, probably not great?
e
It's basically equivalent to using
--no-pantsd
- it forced the daemon to restart. The total wall time of
rm -rf .pids && ./pants X
should be >=
./pants --no-pantsd X
And, reading a bit more - Phoronix is a very good source for Linux news - things look pretty bad for ntfs still: https://www.phoronix.com/news/NTFS3-Linux-Driver-2022-Sad
l
Ah I see, yeah that would not be great then.
h
@late-ram-12573 Are you using WSL2 just because Pants sort-of works under it, or are there other reasons? In other words, if we had native Windows support, would you be using that?
l
Yes, exactly
h
OK, so the real real problem is lack of native Windows support, which we do want to tackle soon
Did we conclude that this does work if the repo is under the user’s homedir?
e
If under any place not 9p, yes.
It's just the 9p fs mounts that are problematic.
That's why I was running down re-mounting via the kernel ntfs driver.
Rob Pike is everywhere
l
Yes it worked when I moved it to my home dir
e
If you're not familiar with Plan 9 (from outer space), it's worth a coffee-break research read. Pretty nifty OS - have not seen the movie.
l
I've not heard of it before today, I'll definitely check it out, thanks!
h
It has tentacles in a bunch of stuff at/from Google, including, notably, Go, due to Rob Pike’s involvement
l
I'm still relatively junior in my dev experience, but if there is anything I can do to help with the windows support, I'd be happy to assist.
❤️ 1