https://pantsbuild.org/ logo
Join Slack
Powered by
I am really struggling to debug a test failure. I ...
# development
l
I am really struggling to debug a test failure. I have a pants pytest that fails when running some rules, but I can't get ahold of the sandbox where the process fails.
Copy code
E       
E       
E       Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

src/python/pants/engine/internals/scheduler.py:507: ExecutionError
==================================== short test summary info =====================================
FAILED src/python/pants/backend/terraform/goals/deploy_test.py::test_run_terraform_deploy
This is despite the the RuleRunner being set up with
Copy code
rule_runner = RuleRunner(
        ...,
        preserve_tmpdirs=True,
    )
which seemingly sets 
--keep-sandboxes=always
under the hood.
✅ 1
c
have you tried the 
logging
decorator on your test method? That usually helps a lot with getting better logging out: https://github.com/pantsbuild/pants/blob/189d20a527cdf2f651ba7d5071c1e949ea8206f6/src/python/pants/testutil/rule_runner.py#L74
l
No I haven't. going to give it a go, thank you!
👍 1
ah, unfortunately did not help. I got some debug output about invalidation of nodes but still can't see the sandbox
This is the output I am getting currently:
Copy code
pants (debug-py-bootstrap-asdf) $ MODE=debug pants --keep-sandboxes=always test --debug src/python/pants/backend/terraform/goals/deploy_test.py:tests --no-test-timeouts -- -k test_run_terraform_deploy -s

====================================== test session starts =======================================
collected 3 items / 2 deselected / 1 selected                                                    

src/python/pants/backend/terraform/goals/deploy_test.py::test_run_terraform_deploy Preserving rule runner temporary directories at /tmp/RuleRunner.d7qzo9ec.
10:40:55.13 [DEBUG] external invalidation: cleared 0 and dirtied 0 nodes for: {"src/tf/BUILD", "", "src", "src/tf"}
10:40:55.13 [DEBUG] external invalidation: cleared 0 and dirtied 0 nodes for: {"src", "src/tf/main.tf", "src/tf", ""}
10:40:55.13 [DEBUG] external invalidation: cleared 0 and dirtied 0 nodes for: {"src", "src/tf/stg.tfvars", "src/tf", ""}
10:40:55.13 [DEBUG] external invalidation: cleared 0 and dirtied 0 nodes for: {"src", "src/tf/stg.tfbackend", "src/tf", ""}
FAILED

============================================ FAILURES ============================================
___________________________________ test_run_terraform_deploy ____________________________________

rule_runner = RuleRunner(build_root=/private/tmp/RuleRunner.d7qzo9ec/BUILD_ROOT)
standard_deployment = StandardDeployment(files={'src/tf/BUILD': '\nterraform_deployment(\n    name="stg",\n    var_files=["stg.tfvars"],\n  ...ate/tmp/pytest-of-gauthamnair/pytest-41/test_run_terraform_deploy0/.terraform/state.json'), target=Address(src/tf:stg))
tmpdir = local('/private/tmp/pytest-of-gauthamnair/pytest-41/test_run_terraform_deploy0')

    @logging(level=LogLevel.DEBUG)
    def test_run_terraform_deploy(rule_runner: RuleRunner, standard_deployment, tmpdir) -> None:
        """Test end-to-end running of a deployment."""
        rule_runner.write_files(standard_deployment.files)
        with mock_console(rule_runner.options_bootstrapper, stdin_content="yes") as (_, m):
>           result = rule_runner.run_goal_rule(
                Deploy, args=["src/tf:stg", *rule_runner.options_bootstrapper.args]
            )

src/python/pants/backend/terraform/goals/deploy_test.py:23: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/python/pants/testutil/rule_runner.py:427: in run_goal_rule
    exit_code = self.scheduler.run_goal_rule(
src/python/pants/engine/internals/scheduler.py:553: in run_goal_rule
    (return_value,) = self.product_request(
src/python/pants/engine/internals/scheduler.py:582: in product_request
    return self.execute(request)
src/python/pants/engine/internals/scheduler.py:519: in execute
    returns, throws = self._execute(execution_request)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pants.engine.internals.scheduler.SchedulerSession object at 0x1066fb6a0>
throws = [Throw(exc=ProcessExecutionFailure("Process 'Find interpreter for constraints: CPython<4,>=3.7' failed with exit code ...ection.\n', engine_traceback=[('pants.core.goals.deploy.run_deploy', '`experimental-deploy` goal'), ('select', None)])]

    def _raise_on_error(self, throws: list[Throw]) -> NoReturn:
        exception_noun = pluralize(len(throws), "Exception")
        others_msg = f"\n(and {len(throws) - 1} more)\n" if len(throws) > 1 else ""
>       raise ExecutionError(
            f"{exception_noun} encountered:\n\n"
            f"{throws[0].render(self._scheduler.include_trace_on_error)}\n"
            f"{others_msg}",
            wrapped_exceptions=tuple(t.exc for t in throws),
        )
E       pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:
E       
E       Engine traceback:
E         in select
E           ..
E         in pants.core.goals.deploy.run_deploy
E           `experimental-deploy` goal
E       
E       Traceback (most recent call last):
E         File "/private/var/folders/z4/dvm2try15dg7gl1vcc_6pv140000gn/T/pants-sandbox-tBoYQc/src/python/pants/core/goals/deploy.py", line 176, in run_deploy
E           deploy_processes = await MultiGet(
E         File "/private/var/folders/z4/dvm2try15dg7gl1vcc_6pv140000gn/T/pants-sandbox-tBoYQc/src/python/pants/engine/internals/selectors.py", line 374, in MultiGet
E           return await _MultiGet(tuple(__arg0))
E         File "/private/var/folders/z4/dvm2try15dg7gl1vcc_6pv140000gn/T/pants-sandbox-tBoYQc/src/python/pants/engine/internals/selectors.py", line 172, in __await__
E           result = yield self.gets
E       pants.engine.process.ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython<4,>=3.7' failed with exit code 102.
E       stdout:
E       
E       stderr:
E       Could not find a compatible interpreter.
E       
E       No interpreters could be found on the system.
E       
E       
E       
E       Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

src/python/pants/engine/internals/scheduler.py:507: ExecutionError
==================================== short test summary info =====================================
FAILED src/python/pants/backend/terraform/goals/deploy_test.py::test_run_terraform_deploy - pan...
================================ 1 failed, 2 deselected in 24.91s ================================
And I am trying to get into the sandbox that fails. I know where the 
await Get(ProcessResult
is that fails.
c
so it’s lying, or is this the wrong sandbox?
Copy code
Preserving rule runner temporary directories at /tmp/RuleRunner.d7qzo9ec.
l
ah ... I was expecting to see the usual message with the name of the specific sandbox. And turns out it is lying. It did preserve the sandbox somewhere inside this directory. Thank you.
💯 1
oh man, this was a horror story, but I think resolved.
c
yea, wrestling with rule code can be taxing.
l
That was an expensive education in f-strings, str-Enum composition, plus some other quirks
Copy code
>>> from enum import Enum
>>> class AsdfPathString(str, Enum):
...     STANDARD = "<ASDF>"
...     LOCAL = "<ASDF_LOCAL>"
... 
>>> AsdfPathString.STANDARD == "<ASDF>"
True
>>> hash(AsdfPathString.STANDARD) == hash("<ASDF>")
True
>>> str(AsdfPathString.STANDARD) == str("<ASDF>")
False
>>> f"{AsdfPathString.STANDARD}"
'<ASDF>'
>>> str(AsdfPathString.STANDARD)
'AsdfPathString.STANDARD'
b
Dunder format vs duner str?
h
Yeah, when testing Pants using Pants there is an "inner Pants" instance running in some tmpdir, and those sandboxes are separate from the outer Pants run's ones, and hard to get at
2 Views
Open in Slack
PreviousNext
Powered by