https://pantsbuild.org/ logo
Join Slack
Powered by
With some much appreciated help from <@UB2J9BQA0>,...
# plugins
j
With some much appreciated help from @hundreds-father-404, I got a Jupyter Notebook app launched from 
pants jupyter
. Before I tackle merging digests or pex files, I wanted to get the exit code correct. When I run the notebook app then exit with CTRL-C CTRL-C, which is the normal procedure with this version of Jupyter Notebook (i.e. 
$? = 0
), 
pantsd
is killed and the whole thing exits with a exit code 
1
. How can I send Jupyter Notebook a CTRL-C without it causing 
pantsd
to be killed?
đź’Ż 1
❤️ 2
Here is the log output to the terminal:
Copy code
[I 09:53:25.043 NotebookApp] Saving file at /Untitled.ipynb
^C09:53:30.02 [INFO] Sending SIGINT to pantsd with pid 24967, waiting up to 5.0 seconds before sending SIGKILL...
[C 09:53:30.031 NotebookApp] received signal 15, stopping
[I 09:53:30.032 NotebookApp] Shutting down kernels
Interrupted by user:
Sending user interrupt to pantsd
[I 09:53:30.236 NotebookApp] Kernel shutdown: 58afed28-9a93-4a6e-98ef-6d7d10000e9f
h
cc @hundreds-breakfast-49010, this has seen a couple different changes recently Btw, are you using rc0 or b3? I believe rc0 made a change with ctrl-c handling
j
I simplified my 
pants jupyter
so the pex doesn't have an entry point. This drops me into the python repl. When I just do a CTRL-D there, it also errors out.
I'm using 
rc0
.
The repl exit is
Copy code
% ./pants jupyter                                                       cb/chartbeat (CICD-8/pants-v2 :zap:) ip-10-6-1-238
13:55:51.31 [INFO] Pex(digest=<pants.engine.internals.native_engine.PyDigest object at 0x10c95c768>, name='jupyter-py3.pex', python=PythonExecutable(path='/Users/myaccount.g/.pyenv/versions/3.6.9/bin/python3.6', fingerprint='3ecacfd795a12540475b4d0d51c424f73dc8b76f76d357e4dd6a91465c7d8ff1'))
Python 3.6.9 (default, Sep 25 2020, 14:09:40)
[GCC Apple LLVM 12.0.0 (clang-1200.0.32.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>
now exiting InteractiveConsole...
13:55:55.01 [INFO] exit code is 0
13:55:55.02 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 281, in run
    engine_result = self._run_v2()
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 193, in _run_v2
    return self._maybe_run_v2_body(goals, poll=False)
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 215, in _maybe_run_v2_body
    poll_delay=(0.1 if poll else None),
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/init/engine_initializer.py", line 127, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/engine/internals/scheduler.py", line 569, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/engine/internals/scheduler.py", line 533, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),

Exception message: 1 Exception encountered:

Engine traceback:
  in select
  in jupyter.goal.jupyter
Traceback (no traceback):
  <pants native internals>
Exception: jupyter.goal:27:jupyter() returned a result value that did not satisfy its constraints: <class 'jupyter.goal.Jupyter'>
Traceback (most recent call last):
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 281, in run
    engine_result = self._run_v2()
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 193, in _run_v2
    return self._maybe_run_v2_body(goals, poll=False)
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 215, in _maybe_run_v2_body
    poll_delay=(0.1 if poll else None),
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/init/engine_initializer.py", line 127, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/engine/internals/scheduler.py", line 569, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/myaccount.g/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0rc0_py36/lib/python3.6/site-packages/pants/engine/internals/scheduler.py", line 533, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

Engine traceback:
  in select
  in jupyter.goal.jupyter
Traceback (no traceback):
  <pants native internals>
Exception: jupyter.goal:27:jupyter() returned a result value that did not satisfy its constraints: <class 'jupyter.goal.Jupyter'>
h
Generally, for a 
repl
goal, you’ll 
return Jupyter(exit_code=0)
. If any part of the rule fails, such as the Pex failing to be created, then the rule will early return with a non-zero exit code. You’ll only reach 
return Jupyter(exit_code=0)
once the 
interactive_runner.run()
has finished.
Hm, that error looks like you’re not returning an instance of the 
Jupyter()
object?
🤦🏽 1
j
Ok. I am doing the following which is not the correct return type:
Copy code
result = interactive_runner.run(
        InteractiveProcess(
            argv=[pex.name], env=None, input_digest=pex.digest, hermetic_env=False
        )
    )

    # exiting from the above with ctrl-c causes the run to end with exit_code 1
    # never gets to this part.


    exit_code = result.exit_code
    <http://logger.info|logger.info>("exit code is {}".format(exit_code)) # XXX make debug
    return exit_code
sweet. That fixed the repl exit! (and by that I mean 
return Jupyter(exit_code=exit_code)
)
đź’Ż 1
🧠 ⬅️ "TYPES" 🔨
h
so, if you run the jupyter notebook with interactive_runner, you can hit ctrl-c in the terminal and it does what you expect?
j
No.
That still kills 
pantsd
instead of the notebook app.
Copy code
^C14:02:38.00 [INFO] Sending SIGINT to pantsd with pid 43331, waiting up to 5.0 seconds before sending SIGKILL...
[C 14:02:38.016 NotebookApp] received signal 15, stopping
[I 14:02:38.016 NotebookApp] Shutting down kernels
h
interrupt handling is definitely something we need to work on a bit more, I think that's expected behavior right now
j
The notebook seems to get it, too.
Ok.
Will that mean the choot is not going to be cached as much as if the notbook exited cleanly?
h
I don't think that will cause problems wrt caching, but can I see your plugin code in its entirely?
j
You are correct. I just re-ran the command a couple of times and it only did the build step on the first run.
I thought a failed exit might result in the chroot build being marked as invalid.
This is acceptable and I will keep an eye out for the fix so I can test it.
Thank you.
Now to get some 
python_library
targets into my chroot.
h
I thought a failed exit might result in the chroot build being marked as invalid.
Ah, I didn’t give the full nuance yesterday with my comment. So a 
Process
is only cached if the exit code is 0. You’re running a 
Process
when you say 
await Get(Pex, PexRequest)
- it’s abstracted away, but underneath, the rule that creates the Pex is literally running a 
Process
that runs 
pex -o my_pex.pex req1
. It doesn’t matter what comes after, like if the rest of the goal rule fails. So long as the 
Process
to build the Pex succeeds, it’ll be cached. -- Further, an 
InteractiveProcess
is never cached. It wouldn’t be safe to cache because there could be user input, which would change what the end result is. So it doesn’t matter with caching if the Jupyter interactive process succeeds or fails.
👍🏽 1
đź‘– 1
Now to get some python_library targets into my chroot.
Cool. You’ll want to request 
addresses: Addresses
in the 
@goal_rule
signature, which will get you the addresses of everything specified by the user on the command line Then use 
await Get(TransitiveTargets, TransitiveTargetsRequest(addresses))
to pull in the transitive closure
🤩 1
Oh, this isn’t documented: use 
await Get(PythonSourceFiles, PythonSourceFilesRequest)
from 
pants.backend.python.util_rules.python_sources
as a convenience to getting all the relevant source files. You’ll see lots of examples if you grep in the Pants codebase You may want to set 
include_files=True
if you want loose files to be used
j
Do I need a step where I combine the 
PythonSourceFileRequest
and my jupyter_pex?
h
To resolve the requirements, use 
PexFromTargetsRequest.for_requirements()
. Check out 
python/goals/repl.py
and look at the ipython repl to see how we resolve the tool Pex and requirements Pex, then join with 
--pex-path
. Note that the interpreter constraints should match each other
Do I need a step where I combine the PythonSourceFileRequest and my jupyter_pex?
Yes, you use 
await Get(Digest, MergeDigests)
for this. 
repl.py
shows that
I think you’re going to end up being able to mostly cargo-cult 
repl.py
, specifically the ipython rule
📦 1
j
Sweet. Standing on the crates of giants.
đź‘– 1
h
Does Jupyter accept Ctrl-D to exit? I usually use that to exit a repl.
j
I'll try
CTRL-D
does not exit. 
CTRL-\
does exit, but 
pantsd
still gets a SIGQUIT.
h
@jolly-midnight-72759 I think that correctly handling daemons running in the foreground without quitting pantsd is a feature we will have to add
👍🏽 1
Open in Slack
PreviousNext
Powered by