https://pantsbuild.org/ logo
Join Slack
Powered by
Hi everyone! First of all thank you so much for pa...
# general
b
Hi everyone! First of all thank you so much for pantsbuild, we’re using at my company and it is a joy to have a mono repo built with speed and confidence! I’d like to ask for some thoughts (I’m likely doing something wrong or just plain silly). My stack: • Running pantsbuild 2.20 with python 3.7.7 on linux with a private gitlab CI runner on docker executor • Pants cache is shared with the docker executors via a host directory mapping mount (instead of using gitlab CI’s cache: feature because it’s too slow for large caches) • Runner concurrency is set to 1 (only one pants process running per executor) However I’m running into to some random issues that randomly happen and get retried by the CI but are very hard to reproduce: • During pip dependency resolution, I often get: 
Exception caught: (pants.engine.internals.scheduler.ExecutionError)
after it logs 
Completed
. It also errors with 
ProcessExecutionFailure: Process 'Resolving 14 requirements … failed with exit code 1.
and 
exited with -9 and STDERR: None
• Just after running 
./pants package
it randomly crashes with 
Exception: Failed to begin read transaction: Resource temporarily unavailable
- I check ulimits for my docker daemon and most values are unlimited so I’m not sure what other resources could be starved thanks!
p
have you tried to run pants in verbose mode?
h
Hey Ricardo, the -9 is OutOfMemory. I wonder if lowering the global option 
pantsd_max_memory
might help: https://www.pantsbuild.org/docs/reference-global#section-pantsd-max-memory-usage
b
Oh how did I miss that 🤔 Thanks for your replies, I’ll give it a shot!
Actually disabling pantsd seems to have stopped the -9 OutOfMemory altogether!
I do get one random issue (which the CI retries and then succeeds). I ran pants with debug level on:
Copy code
$ ./pants --version
12:19:24.85 [DEBUG] File handle limit is: 100000
12:19:25.16 [DEBUG] specs are: Specs(address_specs=AddressSpecs(literals=(), globs=(), filter_by_global_options=True), filesystem_specs=FilesystemSpecs(includes=(), ignores=()))
12:19:25.16 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
2.2.0
$ ./pants --print-stacktrace lint ::
12:19:26.72 [DEBUG] File handle limit is: 100000
12:19:27.04 [DEBUG] specs are: Specs(address_specs=AddressSpecs(literals=(), globs=(DescendantAddresses(directory=''),), filter_by_global_options=True), filesystem_specs=FilesystemSpecs(includes=(), ignores=()))
12:19:27.04 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
12:19:27.04 [DEBUG] Launching 1 roots (poll=false).
12:19:27.04 [DEBUG] computed 1 nodes in 0.001046 seconds. there are 3 total nodes.
12:19:27.04 [DEBUG] requesting <class 'pants.core.goals.lint.Lint'> to satisfy execution of `lint` goal
12:19:27.04 [DEBUG] Launching 1 roots (poll=false).
12:19:27.04 [DEBUG] Starting: `lint` goal
12:19:27.05 [DEBUG] Starting: Find targets from input specs
12:19:27.12 [DEBUG] Completed: Find targets from input specs
12:19:27.12 [DEBUG] Completed: `lint` goal
12:19:27.12 [DEBUG] computed 1 nodes in 0.078090 seconds. there are 554 total nodes.
12:19:27.12 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 276, in run
  engine_result = self._run_v2(goals)
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 190, in _run_v2
  return self._maybe_run_v2_body(goals, poll=False)
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 212, in _maybe_run_v2_body
  poll_delay=(0.1 if poll else None),
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 132, in run_goal_rules
  goal_product, params, poll=poll, poll_delay=poll_delay
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 563, in run_goal_rule
  self._raise_on_error([t for _, t in throws])
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 527, in _raise_on_error
  wrapped_exceptions=tuple(t.exc for t in throws),
Exception message: 1 Exception encountered:
Engine traceback:
 in select
 in pants.core.goals.lint.lint
 in pants.engine.internals.graph.resolve_targets
 in pants.engine.internals.graph.resolve_unexpanded_targets
 in pants.engine.internals.graph.resolve_addresses_from_specs
 in pants.engine.internals.build_files.addresses_from_address_specs
 in pants.engine.internals.build_files.parse_address_family (src/code/services/data)
Traceback (no traceback):
 <pants native internals>
Exception: Failed to begin read transaction: Resource temporarily unavailable
Traceback (most recent call last):
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 276, in run
  engine_result = self._run_v2(goals)
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 190, in _run_v2
  return self._maybe_run_v2_body(goals, poll=False)
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 212, in _maybe_run_v2_body
  poll_delay=(0.1 if poll else None),
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 132, in run_goal_rules
  goal_product, params, poll=poll, poll_delay=poll_delay
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 563, in run_goal_rule
  self._raise_on_error([t for _, t in throws])
 File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.0_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 527, in _raise_on_error
  wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:
Engine traceback:
 in select
 in pants.core.goals.lint.lint
 in pants.engine.internals.graph.resolve_targets
 in pants.engine.internals.graph.resolve_unexpanded_targets
 in pants.engine.internals.graph.resolve_addresses_from_specs
 in pants.engine.internals.build_files.addresses_from_address_specs
 in pants.engine.internals.build_files.parse_address_family (src/code/services/data)
Traceback (no traceback):
 <pants native internals>
Exception: Failed to begin read transaction: Resource temporarily unavailable
k
We also get this often in CI wen running in a docker executor, I wonder if you managed to find more info?
We are getting a lot of:
Copy code
IntrinsicError: Couldn't find file contents for "BUILD": Failed to begin read transaction: Resource temporarily unavailable
3 Views
Open in Slack
PreviousNext
Powered by