https://pantsbuild.org/ logo
Join Slack
Powered by
Can a plugin turn off "pantsd"? In other words, ca...
# plugins
j
Can a plugin turn off "pantsd"? In other words, can I have 
pants jupyter
be the same as 
pants --no-pantsd jupyter
?
h
I don’t think so, unfortunately. 
--pantsd
is a global option that gets set before any goals are even considered Is jupyter having the same issue as ipython that it doesn’t work with pantsd?
j
A little. I can run it with pantsd, but the output is more "Jupyter-esq" without pantsd. (e.g. ctrl-c asks for confirmation without pantsd and just quits (fails) with pantsd.)
👍 1
h
Okay, Greg is looking into improving that in the background. Sounds like it’s not as bad as IPython is, at least. IPython straight up won’t work with Pantsd 😕
👍🏽 1
h
@jolly-midnight-72759 do you have the source code for your plugin that runs 
jupyter
in a publicly-readable form? I'm interested in seeing exactly how you're running 
jupyter
right now
j
I can share it with you, but can't get to it until later this afternoon.
h
no problem, I'd love to see it whenever you have a chance
j
@hundreds-breakfast-49010 Here is the current partially working of our jupyter pants v2.0.0 plugin: https://gist.github.com/rcuza/bbcf8f5a13964db7760ff376649c1a44
h
thanks @jolly-midnight-72759
j
Please excuse the "construction dust". This plugin is not fully functional. 🙂
h
no worries, I wanted to see this so I could see how you were running jupyter. that makes it easier to figure out how to handle sending keyboard interrupts to it correctly
👍🏽 1
j
This is also an older version of jupyter. The newer versions use 
subprocess.Popen
to launch the app and each kernel separately. Works great in a venv but I don't know how to get it working with pex's.
h
@jolly-midnight-72759 if you have some time, could you explain how the 
jupyter_stubber
entry point code is meant to work? it looks like "jupyter_stubber" might be one of your org's custom packages?
I'm basically trying to run the jupyter daemon in my own pants environment at the moment
h
It’s a small script that acts as a custom launcher, iiuc. Similar to how we use a custom launcher for MyPy. I think it’s nothing more than 
jupyter_stubber_run
🤔 1
h
ok, cool, I can launch jupyter locally
💯 1
and replicate the issue with ctrl-c also killing pantsd (although we expected that)
j
Sweet. @hundreds-breakfast-49010 did you make a separate installable module like we do with Jupyter Stubber? Or did you launch it from the function in 
goal.py?
If you launch the plugin with 
--no-pantsd
, then the Jupyter app will catch the first ctrl-c similar to when run in a venv.
h
@jolly-midnight-72759 my code is here: https://github.com/gshuflin/pants/commit/d9b5baf3d263e8c99d20c91d85973da8bfed3d34 , based on your snippet. I rigged up something similar to the 
LAUNCHER_FILE
construct we use in https://github.com/pantsbuild/pants/blob/master/src/python/pants/backend/python/typecheck/mypy/rules.py#L166 based on eric's earlier suggestion
💯 2
j
I like the 
LAUNCHER_FILE
better than the stubber solution. I got it working on my repo, too.
How can I get the target on the PYTHONPATH? I want to be able to 
import
and use any python code in the target's sources along with its dependencies. So if I run 
pants jupyter project/src/python/helloworld
, I can do 
import helloworld
in a Jupyter Notebook. Do I have to package the target and then add it to the requirements?
Weird. I just picked a different target and it worked but not for its dependencies.
h
the code in that snippet is creating 
merged_digest
from merging the digest of the 
Pex
containing the jupyter executable, and the digest of all the source files associated with the targets passed into the 
jupyter
goal
and then passes that 
merged_digest
to 
InteractiveProcessRunner
so the file system within the runner will have 
jupyter-py3.pex
in it, and then a tree representing the subset of the source code specified by the targets and their dependencies
and this is what the jupyter file browser shows when it runs
PYTHONPATH
is already set within the jupyter environment, I think based on the 
LAUNCHER_FILE
code that uses 
sys.path
. but 
sys.path
was already initialized by whatever the 
PYTHONPATH
would've been when the pex started running
actually now that I'm looking closely at it, I'm not sure exactly what that 
os.environ["PYTHONPATH"] = ":".join(sys.path)
line is doing
as in, I'm not sure why it's necessary
I would think that whatever changes to the 
PYTHONPATH
are necessary to get the right import behavior when running a notebook under jupyter could be done here in this script, but I haven't tried this myself yet
j
The 
sys
stuff is copies for the stubber that was needed for v1. Maybe it isn’t needed in v2. See the note in the code explaining why we did it that way.
(Basically in v2, it is adding all the pex paths to 
PYTHONPATH
.) when I’m at my keyboard next, I’ll see if the jupyter kernel starts without it.
Starting a new thread...
2 Views
Open in Slack
PreviousNext
Powered by