https://pantsbuild.org/ logo
#general
Title
# general
l

late-ram-12573

08/17/2022, 1:35 PM
Hi Everyone, I'm just getting started with pants, trying to move my team to a monorepo. I've been experimenting with setting up a test project to get a feel for pants, but my updates to files are not getting picked up. For example, my source root pattern is "src/*". if I run pants roots, I see 3 directories I made, A, B, and C. If I add directory D and rerun pants roots, I still only see A, B, and C. I noticed that if I make an adjustment to my pants.toml file, it reinitializes the scheduler and the changes are then picked up. After making an adjustment and rerunning pants roots, I now see A, B, C and D. This is just an example with roots, but the same issue is happening with any changes I make to BUILD files, which is incredibly frustrating. Has anyone else experienced this? I'm currently on Windows 10 and using WSL
👋 2
w

wide-midnight-78598

08/17/2022, 1:37 PM
That seems very strange, I personally haven't run into this, but maybe someone else has. In the meantime, is this something you could create a github sample for, or share your pants.toml/other code? I could see if I can repro on my machine
l

late-ram-12573

08/17/2022, 1:46 PM
w

wide-midnight-78598

08/17/2022, 1:46 PM
I just tried on my Mac - and
roots
was picking up the new folders on every command, without any funny business, I'll grab your repo and try now
l

late-ram-12573

08/17/2022, 1:47 PM
Thank you for checking this out with me!
w

wide-midnight-78598

08/17/2022, 1:49 PM
Copy code
sj@tinyrick pants_exploration % ./pants roots  
src/lib
src/projects
src/tools

sj@tinyrick pants_exploration % mkdir src/helloworld

sj@tinyrick pants_exploration % ./pants roots       
src/helloworld
src/lib
src/projects
src/tools
Seems to work fine - I wonder if this is a WSL2 thing? Or, if maybe there is some strange
AllRoots
caching, but only on windows?
l

late-ram-12573

08/17/2022, 1:50 PM
Must be WSL2, it's not just the roots, but any changes I make, including BUILD files
I'll continue to search around, I appreciate the help!
w

wide-midnight-78598

08/17/2022, 1:50 PM
🤯
Would you mind trying something out?
./pants -ldebug roots
mkdir src/123456
./pants -ldebug roots
Curious if the debug logs shows anything, both "before" and "after" a change
One other idea I'd be curious about... In your pants.toml, maybe try stopping the pants daemon from running and see if that makes a difference:
Copy code
[GLOBAL]
pantsd = false
l

late-ram-12573

08/17/2022, 1:57 PM
aha, turning off pantsd worked
w

wide-midnight-78598

08/17/2022, 1:58 PM
Hmm, that's both good and unfortunate. This was created yesterday, is this yours? https://github.com/pantsbuild/pants/issues/16534
l

late-ram-12573

08/17/2022, 1:59 PM
Not me, but that's the exact issue I am having.
w

wide-midnight-78598

08/17/2022, 1:59 PM
This was also referenced: https://github.com/pantsbuild/pants/issues/10445 I'll make an update to that issue - but maybe see if the home directory workaround works with the pants daemon on?
l

late-ram-12573

08/17/2022, 2:01 PM
sorry, what do you mean by home directory workaround
w

wide-midnight-78598

08/17/2022, 2:01 PM
https://github.com/pantsbuild/pants/issues/10445 This one: "Moving the repository in the users home directory (
~
) fixes the issue."
That was pants 1, but worth seeing if the behaviour is similar
l

late-ram-12573

08/17/2022, 2:06 PM
Yes, moving to home and re-enabling pantsd worked
w

wide-midnight-78598

08/17/2022, 2:09 PM
🤯 🤯 🤯 🤯
Unfortunately, as I don't know how the daemon works, nor have I used WSL in quite some time, I'm not much more help. My guess is around permissions, but 🤷
e

enough-analyst-54434

08/17/2022, 2:17 PM
@late-ram-12573 what does
df -h
say? IIUC WSL2 uses, of all things, 9p for mounting Windows drives / folders. That's a network filesystem (from Plan 9!!) and network filesystems pretty famously don't work with many more advanced fs operations like locking and event notifications in general (NFS v4 I think finally mostly does work with locking, not sure about fs events).
l

late-ram-12573

08/17/2022, 2:20 PM
Sorry, not sure what to look for here:
w

wide-midnight-78598

08/17/2022, 2:20 PM
In reading more about WSL2, it seems a lot of people are having problems of the "file changed" variety due to inotify events not being sent up, or other reasons. Web dev seems hit hard in particular
e

enough-analyst-54434

08/17/2022, 2:21 PM
@late-ram-12573 it's the 1st filesystem column which Windows unhelpfully productizes and just says C:\ - which is not a filesystem 🤦
But, pretty sure it's 9p and that's the issue here.
Ah, my bad actually. How about
df -T
?
That should give a type column.
l

late-ram-12573

08/17/2022, 2:27 PM
image.png
using /mnt/d for my project, so it is 9p
e

enough-analyst-54434

08/17/2022, 2:33 PM
Aha, ok. That is definitely it then. Pants really doesn't work with network filesystems.
I think everyone who uses Pants regularly can agreed
--no-pantsd
is not really any sort of solution at all. FS event handling is critical to perf.
l

late-ram-12573

08/17/2022, 2:36 PM
That's unfortunate, but I certainly appreciate the help.
e

enough-analyst-54434

08/17/2022, 2:39 PM
It is. Really we get lucky WSL exists at all since we've never expended the effort to get Pants and its backends all working specifically on Windows. All the developers back through 2010 inception have been on Linux or MacOS and so this support has never come.
I know there has been recent mainlining of the Paragon NTFS driver into the Linux kernel. I wonder if your kernel is new enough / can be upgraded to one where you can just re-mount as ntfs?
@late-ram-12573 what does
cat /proc/filesystems
say? Any ntfs in there?
l

late-ram-12573

08/17/2022, 2:47 PM
just nfs
image.png
e

enough-analyst-54434

08/17/2022, 2:49 PM
Yeah - ok.
Have you tried
apt update && apt install ntfs-3g
?I think WSL(2) defaults to an Ubuntu / Debian image.
Ah, sorry - wrong package. That's the Fuse-based impl. May work, but should be slow.
l

late-ram-12573

08/17/2022, 2:57 PM
I can try that but I do have a question for you. In the issues https://github.com/pantsbuild/pants/issues/16534 the opener mentioned deleting the .pids folder before running pants commands worked. I just tried it and it does for me as well. It's a "hacky" solution, but I added a command to delete the .pids folder when running the pants shell script, and it seems to work. Is this a terrible idea?
h

happy-kitchen-89482

08/17/2022, 3:00 PM
I think that might cause pantsd to restart on every run… ? So, probably not great?
e

enough-analyst-54434

08/17/2022, 3:01 PM
It's basically equivalent to using
--no-pantsd
- it forced the daemon to restart. The total wall time of
rm -rf .pids && ./pants X
should be >=
./pants --no-pantsd X
And, reading a bit more - Phoronix is a very good source for Linux news - things look pretty bad for ntfs still: https://www.phoronix.com/news/NTFS3-Linux-Driver-2022-Sad
l

late-ram-12573

08/17/2022, 3:02 PM
Ah I see, yeah that would not be great then.
h

happy-kitchen-89482

08/17/2022, 3:02 PM
@late-ram-12573 Are you using WSL2 just because Pants sort-of works under it, or are there other reasons? In other words, if we had native Windows support, would you be using that?
l

late-ram-12573

08/17/2022, 3:03 PM
Yes, exactly
h

happy-kitchen-89482

08/17/2022, 3:04 PM
OK, so the real real problem is lack of native Windows support, which we do want to tackle soon
Did we conclude that this does work if the repo is under the user’s homedir?
e

enough-analyst-54434

08/17/2022, 3:07 PM
If under any place not 9p, yes.
It's just the 9p fs mounts that are problematic.
That's why I was running down re-mounting via the kernel ntfs driver.
Rob Pike is everywhere
l

late-ram-12573

08/17/2022, 3:09 PM
Yes it worked when I moved it to my home dir
e

enough-analyst-54434

08/17/2022, 3:10 PM
If you're not familiar with Plan 9 (from outer space), it's worth a coffee-break research read. Pretty nifty OS - have not seen the movie.
l

late-ram-12573

08/17/2022, 3:11 PM
I've not heard of it before today, I'll definitely check it out, thanks!
h

happy-kitchen-89482

08/17/2022, 3:11 PM
It has tentacles in a bunch of stuff at/from Google, including, notably, Go, due to Rob Pike’s involvement
l

late-ram-12573

08/17/2022, 3:13 PM
I'm still relatively junior in my dev experience, but if there is anything I can do to help with the windows support, I'd be happy to assist.
❤️ 1
7 Views