Hi everyone First of all thank you so much for pantsbuild we

Question

Hi everyone First of all thank you so much for pantsbuild we re using at my company and it is a joy to have a mono repo built with speed and confidence I d like to ask for some thoughts I m likely doing something wrong or just plain silly My stack Running pantsbuild 2 20 with python 3 7 7 on linux with a private gitlab CI runner on docker executor Pants cache is shared with the docker executors via a host directory mapping mount instead of using gitlab CI s cache feature because it s too slow for large caches Runner concurrency is set to 1 only one pants process running per executor However I m running into to some random issues that randomly happen and get retried by the CI but are very hard to reproduce During pip dependency resolution I often get `Exception caught pants engine internals scheduler ExecutionError ` after it logs `Completed` It also errors with `ProcessExecutionFailure Process Resolving 14 requirements failed with exit code 1 ` and `exited with 9 and STDERR None` Just after running ` pants package` it randomly crashes with `Exception Failed to begin read transaction Resource temporarily unavailable` I check ulimits for my docker daemon and most values are unlimited so I m not sure what other resources could be starved thanks

plain-sundown-25537 · Answer

have you tried to run pants in verbose mode

hundreds-father-404 · Answer

Hey Ricardo the 9 is OutOfMemory I wonder if lowering the global option `pantsd max memory` might help <https www pantsbuild org docs reference global section pantsd max memory usage>

boundless-salesclerk-97604 · Answer

Oh how did I miss that thinking face Thanks for your replies I ll give it a shot

boundless-salesclerk-97604 · Answer

Actually disabling pantsd seems to have stopped the 9 OutOfMemory altogether

boundless-salesclerk-97604 · Answer

I do get one random issue which the CI retries and then succeeds I ran pants with debug level on

boundless-salesclerk-97604 · Answer

```$ pants version 12 19 24 85 DEBUG File handle limit is 100000 12 19 25 16 DEBUG specs are Specs address specs=AddressSpecs literals= globs= filter by global options=True filesystem specs=FilesystemSpecs includes= ignores= 12 19 25 16 DEBUG changed options are ChangedOptions since=None diffspec=None dependees= lt DependeesOption NONE none gt 2 2 0 $ pants print stacktrace lint 12 19 26 72 DEBUG File handle limit is 100000 12 19 27 04 DEBUG specs are Specs address specs=AddressSpecs literals= globs= DescendantAddresses directory= filter by global options=True filesystem specs=FilesystemSpecs includes= ignores= 12 19 27 04 DEBUG changed options are ChangedOptions since=None diffspec=None dependees= lt DependeesOption NONE none gt 12 19 27 04 DEBUG Launching 1 roots poll=false 12 19 27 04 DEBUG computed 1 nodes in 0 001046 seconds there are 3 total nodes 12 19 27 04 DEBUG requesting lt class pants core goals lint Lint gt to satisfy execution of `lint` goal 12 19 27 04 DEBUG Launching 1 roots poll=false 12 19 27 04 DEBUG Starting `lint` goal 12 19 27 05 DEBUG Starting Find targets from input specs 12 19 27 12 DEBUG Completed Find targets from input specs 12 19 27 12 DEBUG Completed `lint` goal 12 19 27 12 DEBUG computed 1 nodes in 0 078090 seconds there are 554 total nodes 12 19 27 12 ERROR Exception caught pants engine internals scheduler ExecutionError File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants bin local pants runner py line 276 in run engine result = self run v2 goals File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants bin local pants runner py line 190 in run v2 return self maybe run v2 body goals poll=False File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants bin local pants runner py line 212 in maybe run v2 body poll delay= 0 1 if poll else None File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants init engine initializer py line 132 in run goal rules goal product params poll=poll poll delay=poll delay File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants engine internals scheduler py line 563 in run goal rule self raise on error t for t in throws File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants engine internals scheduler py line 527 in raise on error wrapped exceptions=tuple t exc for t in throws Exception message 1 Exception encountered Engine traceback in select in pants core goals lint lint in pants engine internals graph resolve targets in pants engine internals graph resolve unexpanded targets in pants engine internals graph resolve addresses from specs in pants engine internals build files addresses from address specs in pants engine internals build files parse address family src code services data Traceback no traceback lt pants native internals gt Exception Failed to begin read transaction Resource temporarily unavailable Traceback most recent call last File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants bin local pants runner py line 276 in run engine result = self run v2 goals File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants bin local pants runner py line 190 in run v2 return self maybe run v2 body goals poll=False File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants bin local pants runner py line 212 in maybe run v2 body poll delay= 0 1 if poll else None File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants init engine initializer py line 132 in run goal rules goal product params poll=poll poll delay=poll delay File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants engine internals scheduler py line 563 in run goal rule self raise on error t for t in throws File cicache cache pants setup bootstrap Linux x86 64 2 2 0 py37 lib python3 7 site packages pants engine internals scheduler py line 527 in raise on error wrapped exceptions=tuple t exc for t in throws pants engine internals scheduler ExecutionError 1 Exception encountered Engine traceback in select in pants core goals lint lint in pants engine internals graph resolve targets in pants engine internals graph resolve unexpanded targets in pants engine internals graph resolve addresses from specs in pants engine internals build files addresses from address specs in pants engine internals build files parse address family src code services data Traceback no traceback lt pants native internals gt Exception Failed to begin read transaction Resource temporarily unavailable```