https://pantsbuild.org/ logo
Join Slack
Powered by
Question about porting a v1 jupyter plugin: 1. Sin...
# plugins
j
Question about porting a v1 jupyter plugin: 1. Since it is spawning a Jupyter Notebook, I don't need to use 
InteractiveProcess
, correct? All the interaction will be happening in the browser. 2. Does using 
run_in_workspace=True
make the most sense for making the target code accessible to the notebook or should I use a digest?
h
Jupyter is interesting. How did you implement it with v1? My understanding of Jupyter is that it starts a daemon, right? How are you doing cleanup of that daemon with v1 right now?
h
InteractiveProcess
was created to support cases where a process that pants executes needs non-cached, live access to the console - basically repls, and running a test in a mode where you can spawn pdb (which is itself sort of a repl). 
Process
on the other hand is designed for process whose results can be cached
👍🏽 1
I'm not sure if either of these is necessarily correct for a daemon like jupyter-notebook
we're definitely interested in the details of how your plugin works. it's possible that we will need to adjust the semantics of one of those types, or even introduce some kind of new daemon type, to support things like having pants spawn a jupyter daemon
@hundreds-father-404 my (limited) understanding of jupyter is that you typically run it from the console in the form 
$ jupyter notebook
, which then starts a daemon running in your console, and opens up a page in your browser at 
localhost:<some port>
, presenting you with a file tree view of the local directory. then you pick whatever notebook you want to work with in the browser UI, and work with it there. the web page is talking back-and-forth with the daemon running live in your console, and the javascript will pop up some kind of warning and things will break if you kill that daemon but leave the page up
👍 2
j
Yes. Our plugin allows us to use the code in our monorepo with a jupyter notebook in a pants way. So we run 
pants jupyter aioverlord/src/python/ownfb:spicymemes
. This will let us import the 
spicymemes
library and all its dependences as defined in the 
BUILD
file of the target. The Jupyter Notebook opens at the top of the repo (its root).
h
And then does Pants keep running in the foreground until you shut down the notebook?
j
yes. The Jupyter Notebook app logs to the console while it is running. When you control-c, it closes down the app and does the 
pants
shutdown steps.
👍 1
I don't know if it is important that 
pants
be running while Jupyter Notebook is running.
h
Okay, cool, then yeah, I think 
InteractiveProcess
will work. I imagine that the deamon is in the background, but there is still a process in the foreground that is writing to the terminal, and that shutting down the foreground process shuts down the daemon. That works well with Pants. 
InteractiveProcess
will run that foreground process for you, and then ctrl-c will shut down the foreground process and trigger Jupyter to clean up its daemon, iiuc
As mentioned in the other thread, if you use the plugin hook for the 
repl
goal, it will already set up the 
InteractiveProcess
for you. Otherwise, with your own Jupyter goal, you would set that up yourself
j
Based on what I learned from the other thread, I can save myself some unnecessary work by hooking into 
core/goals/repl.py
. If I do that, can I still make a custom goal so users can run 
pants jupyter
instead of 
pants repl --shell=jupyter
like they are used to doing now?
I believe I will need a 
@goal_rule
decorated class if this can be accomplished.
h
Yes, you can implement both ways :) and you can factor out a common rule so that there’s less duplication of code. My advice is to start by only doing one implementation, though, for simplicity Yes, that’s correct with using a 
@goal_rule
2 Views
Open in Slack
PreviousNext
Powered by