https://pantsbuild.org/ logo
#general
Title
# general
p

plain-sundown-25537

11/16/2020, 2:57 AM
Is pants doing something on ulimit_n on Mac OS?
Copy code
❯ ipython
Python 3.7.5 (default, Dec 26 2019, 04:25:18)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import jedi
In [2]: import resource
In [3]: resource
Out[3]: <module 'resource' from '/Users/leozc/.pyenv/versions/3.7.5/lib/python3.7/lib-dynload/resource.cpython-37m-darwin.so'>
In [4]: resource.getrlimit(resource.RLIMIT_NOFILE)
Out[4]: (65536, 9223372036854775807) 
in sandbox
While kick the run in pants
Copy code
❯ ./pants repl helloworld/rayburst.py #my own app
Python 3.7.5 (default, Dec 26 2019, 04:25:18)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import resource
>>> resource.getrlimit(resource.RLIMIT_NOFILE)
(9223372036854775807, 9223372036854775807)
❯ ./pants --version 185817.15 [INFO] initializing pantsd... 185817.89 [INFO] pantsd initialized. 2.1.0rc0
w

witty-crayon-22786

11/16/2020, 6:02 PM
we attempt to increase the ulimit if we can, yea
p

plain-sundown-25537

11/16/2020, 10:54 PM
rlimit:Resource:NOFILE.set(max, max) ^^ should we be more conservative? The number above is way too big IMHO, also some app using the
min
to guess connection count see here
w

witty-crayon-22786

11/16/2020, 10:59 PM
mm. interesting. yea, we could be more conservative. we know that 10k is a safe value for pants, so we could avoid setting it higher than
min(current_value, 10k)
p

plain-sundown-25537

11/16/2020, 11:00 PM
pls see above the number above
Copy code
resource.getrlimit(resource.RLIMIT_NOFILE)
(9223372036854775807, 9223372036854775807)
on mac os
I saw your code.. and not sure why the number came out so big
w

witty-crayon-22786

11/16/2020, 11:02 PM
@plain-sundown-25537: the two values are the “soft” and “hard” limits, respectively
p

plain-sundown-25537

11/16/2020, 11:03 PM
yes - the soft and hard both very big - see above
w

witty-crayon-22786

11/16/2020, 11:03 PM
if the hard limit is that high, it’s because someone with root has configured it that way on your machines
p

plain-sundown-25537

11/16/2020, 11:03 PM
I was on Macos Big Sur
HmmMmm~
w

witty-crayon-22786

11/16/2020, 11:03 PM
ah. i wonder if they changed the default then
p

plain-sundown-25537

11/16/2020, 11:04 PM
That is what I am thinking too, I don’t think anyone would ever come up such big number
w

witty-crayon-22786

11/16/2020, 11:05 PM
ok… i can make a quick change to do roughly
soft_limit = min(hard_limit, 10k)
👍 2
i think that redis library should probably react to your PR though.
p

plain-sundown-25537

11/16/2020, 11:08 PM
IMHO both sides should do something - Ray needs to be more robust, Pants needs to avoid the number blowing up too much
w

witty-crayon-22786

11/16/2020, 11:33 PM
@plain-sundown-25537: will you folks need this backported to 1.30.x, or is 2.0.x sufficient?
p

plain-sundown-25537

05/06/2021, 3:45 PM
2.0.x is enough.. sorry for having this answered 6 months late
7 Views