Any suggestion on how to increase permanently the ...
# general
c
Any suggestion on how to increase permanently the
ulimit
with
WSL2
I can't make it work correctly on Windows, every time I get, I also tried to modify different files following many tutorial, I still have the warning at the beginning...
Copy code
ulimit -n 65536
-bash: ulimit: open files: cannot modify limit: Operation not permitted
e
WSL2 is not specific enough, its really down to the Linux distribution you use. Which is that? And systemd or no? I think WSL2 supports systemd.
c
Ubuntu 22.04 LTS
e
Ok, just a sec.
Ok, I'm also WSL2 Ubuntu 22.04 for most of my work (20.04 sometimes). For 22.04 I am using stock, no edits to /etc/security/limits.conf or ~/.bashrc bumps. I have no issues running Pants ... so what files have you tried then?
My ulimit fwiw:
Copy code
$ ulimit -n
1024
c
so far tried the classic
ulimit
command like describe above, I have tried to add my user to
/etc/security/limits.conf
but without luck (still at 1024) and also tried
Copy code
sudo prlimit -p "$$" --nofile=10000:10000
exec zsh
ulimit -n 10000
which work but only for the running session
@enough-analyst-54434 I have over 30 Projects with over 10k files and when I run the whole tests suite I crash building a PEX with the same errors as described in the documentation for files limits
e
I just added that new line at end of file:
Copy code
$ whoami
jsirois
$ tail -5 /etc/security/limits.conf
#@student        -       maxlogins       4

# End of file

jsirois         hard    nofile           65536
Then over in powershell:
Copy code
> wsl -t Ubuntu-22.04
The operation completed successfully.
Then opened a new Ubuntu-22.04 session:
Copy code
$ ulimit -n
1024
$ ulimit -n 65536
$ ulimit -n
65536
$ ulimit -n 65537
-bash: ulimit: open files: cannot modify limit: Operation not permitted
A similar `/etc/security/limits.conf`edit doesn't work for you?
c
@enough-analyst-54434 thanks now I can run correctly
ulimit -n 65536
but I still have this warning
11:44:23.78 [WARN] File handle limit is capped to: 4096. To avoid 'too many open file handle' errors, we recommend a limit of at least 10000: please see <https://www.pantsbuild.org/docs/troubleshooting#too-many-open-files-error> for more information.
when running Pants
e
And you
pkill pantsd
or however you ensure Pantsd is down and then re-started it afresh after you bumped to 65K?
c
@enough-analyst-54434 I realized I still can't exec
ulimit -n 65536
with my user (I was typing
ulimit 65536
without the
-n
) so no, even with the edit of my
/etc/security/limits.conf
I can't modify (restarted WSL, restarted my pc, wipe all pants cache just to be sure)
Copy code
cerberus@LAPTOP-BASTIEN:/mnt/c/projects/pant$ tail -5 /etc/security/limits.conf
#ftp             hard    nproc           0
#ftp             -       chroot          /ftp
#@student        -       maxlogins       4
# End of file
cerberus        hard    nofile  65536
cerberus@LAPTOP-BASTIEN:/mnt/c/projects/pant$ ulimit -n 65536
bash: ulimit: open files: cannot modify limit: Operation not permitted
e
Huh, not sure what to say about that.
ANy hits on
ps -ef | grep systemd
?
c
No stress, anyway it work in the pipeline correctly, that was just for local development and I understand the problem is not coming from Pants but from WSL and Windows...
Copy code
ps -ef | grep systemd
cerberus  1295  1286  0 13:17 pts/0    00:00:00 grep --color=auto systemd
e
Ok ~good, but even more mysterious then since you and I have the superficially same setup and I do not have your experience.
Copy code
$ id
uid=1000(jsirois) gid=1000(jsirois) groups=1000(jsirois),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),119(lxd),121(wireshark),999(docker)
Are you in adm? I did not set that up explicitly for my user, but may be relevant.
c
yup I am in adm
Copy code
id
uid=1000(cerberus) gid=1000(cerberus) groups=1000(cerberus),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),117(netdev),1001(docker)
e
Ok, no clue really.
So ... ulimit is a bash builtin. You're using zsh?
c
nope pure bash
e
K, truly out of ideas then.
c
it works for the zsh session only doing it that way
e
Ok.
c
but no result for bash
and I was trying to set it permanenet so no need to sudo stuff when running the script
e
which bash
? Because, very weird to me but:
Copy code
jsirois@Gill-Windows:~/dev/pantsbuild/jsirois-scie-pants (one_step_setup/parity) $ which bash
/usr/bin/bash
jsirois@Gill-Windows:~/dev/pantsbuild/jsirois-scie-pants (one_step_setup/parity) $ /mnt/c/Program\ Files/Git/bin/bash.exe

John Sirois@Gill-Windows MINGW64 //wsl.localhost/Ubuntu-22.04/home/jsirois/dev/pantsbuild/jsirois-scie-pants
$
IOW the Git bash.exe Windows executable works under WSL. I don't really understand WSL. And Git bash fails:
Copy code
John Sirois@Gill-Windows MINGW64 //wsl.localhost/Ubuntu-22.04/home/jsirois/dev/pantsbuild/jsirois-scie-pants
$ ulimit -n
3200

John Sirois@Gill-Windows MINGW64 //wsl.localhost/Ubuntu-22.04/home/jsirois/dev/pantsbuild/jsirois-scie-pants
$ ulimit -n 10000
bash: ulimit: open files: cannot modify limit: Too many open files

John Sirois@Gill-Windows MINGW64 //wsl.localhost/Ubuntu-22.04/home/jsirois/dev/pantsbuild/jsirois-scie-pants
$
c
launching Ubuntu with
ubuntu run bash
and I get
/usr/bin/bash
on the
which bash
e
Ok. That was the only way I could repro a ulimit set failure at any rate, using Windows bash.exe under WSL2 Ubuntu-22.04.