Hey there! Any experience on how to have a functioning jupyter notebook integration - we use them a lot to iterate over the code rather than static python files I found this plugin: https://github.com/pantsbuild/pants-jupyter-plugin But having some troubles setting it up. What I am trying to achieve is to basically load into the notebook the requirements for a specific resolve or combination of resolves (eg: how it's done with

parametrize

) - or in other words it would be great to have the jup notebook that loads dependencies like pants run, so eg when I execute:

pants run kittl/projects/my_project/main.py

I tried adding to my BUILD `pex_binary`:

python_sources(
    name="my_project",
    resolve="my_project",
)

python_requirements(
    name="reqs",
    source="requirements.txt",
    resolve="my_project",
)

pex_binary(
    name="my_project_bin"
)

And loading with the magic:

%pants_load kittl/projects/my_project

But it fails because it can't find the right version of python in the path of the notebook, specifically it's not looking at the

pyenv root

folder Would you have any suggestion to fix this? Thanks so much! Really appreciate the help! 🙂

If you're willing to wait a few weeks I can see about open-sourcing my Jupyter Notebooks plugin. At $work we use it to

pants run path/to/notebook.ipynb

Sounds great! Does this run the jup notebook sequentially cell by cell or injects the dependencies into the environment so that you can run interactive sesssion with it?

The latter

That would be amazing to have!

Can you see if there's a relevant GitHub Issue? If not, can you create one? Put the link here and I'll assign myself (Sorry I'm on mobile, so i'm a bit unable to do complex tasks(

no worries at all! will do by the end of the day, thanks so much 🙂

Curious In the meanwhile what can be done. Could one add a separate requirements with just Jupyter associated to the same resolve and make a PEX that depends on it as well as the sources with the Jupyter command as entry point?

Sounds good @late-advantage-75311! Would it be much trouble to provide a minimal working example?

I will give it a crack in a few hours and ping back

Thanks so much!

nice, this looks like it is working. making an example repo ...

That's fantastic, thanks so much! 🙂

Here it is! https://github.com/gauthamnair/pants-jupyter-example

Amazing will check it tomorrow, thanks a lot dear @late-advantage-75311 !

I'd been meaning to explore how one would get this to work. There are def some complications to a solution of this kind, because it does require restarting the server any time you want to bring in new dependency code (either 3rd party or your own), so it doesn't really help if you are doing something like interactively working with a library you are developing on one side and a notebook to play with it on the other side. For those situations where there is tight iterative coupling between development in the lib code and testing in the notebook it may be better to take an approach like what one does with IDE setup (jupyter becoming literally the development environment in this case): have pants export a virtual environment, setup PYTHON_PATH appropriately, and start a jupyter notebook. Then one could iterate and periodically

importlib.reload

library modules.

I see! That's actually the case for our company, the iterative development - the idea sounds really good! I'm terribly sorry to bother, but if you have any time would it be much trouble setting up a minimal example for this setup as well? I believe it can be very helpful to many people as this interactive development workflow is really common!

oh, that doesn't require a repo, you could do it in this same repo pretty easily ... let's see ...

Amazing dear @late-advantage-75311 thank you so much for real, looks great! Will try it out tomorrow morning and let you know 🙂 many thanks again!

that's odd, I don't have much experience with VSCode, but I have used the export command to generate python interpreters for use with pycharm. Maybe a VSCode user may be able to chime in.

That's very strange indeed, i actually managed to solve by removing cache - but that's fantastic, now i can use vscode jupyter integration 🙂 thanks so much once again! super appreciative of your help and time dear Gautham!

And thank you for the autoreload extension tip. Didn't know about it

My pleasure dear Gautham! 🙂