<#23652 Pants will spawn more threads than LMDB ca...
# github-notifications
q
#23652 Pants will spawn more threads than LMDB can handle on large machines Issue created by svenier Pants defaults to (num_cores / 2) for rule_threads_core, and a "small multiple" of that for rule_threads_max, plus num_cores for process_execution_local_parallelism. When running on a large compute farm machine with 128 cores, that results in a very large number of concurrent things happening at once. LMDB has a limit of 126. And given enough attempts, we eventually lose the race and Pants crashes with:
IntrinsicError: An error occurred when attempting to materialize a working directory at "/tmp/pants-sandbox-dp96H8": Failed to begin read transaction: MDB_READERS_FULL: Environment maxreaders limit reached
This has been observed with multiple versions, let's call it 2.32.0, running on Linux. The workaround is to set reasonable sizes for those concurrency limits, which we need to do anyway since these machines are shared resources and we can't allow a single user to monopolize them. But it would still be good if by default Pants did not exceed the capabilities of LMDB. pantsbuild/pants