Running into a separate issue for jupyter-book, so starting a separate thread: when I try to run

jupyter-book

on a jupyter notebook (rather than markdown) the pex doesn't have ipykernel properly setup. The issue is documented here: github.com/jupyter/notebook/issues/2636 I don't have a lot of familiarity with the jupyter kernel, but it looks like the issue is that the kernel isn't properly set up in the sandbox. Looking for anyone who's found a solution? I tried using the

pants-jupyter-plugin

to no avail.

That issue has likely 0 relation to modern Pex - it is very old. In particular,

--venv

PEXes are new since then and install themselves as a standard venv before running the 1st time. You may need to specify

--venv-site-packages-copies

, but with those two options a PEX should be pretty much indistinguishable from a non PEX. This was written from a Pex perspective and not a Pants perspective. I like to disentangle the two when debugging issues like this. Have you experiment with venv knobs, etc?

not sure what that is. I'm new to the whole world of pants and pex.

Right, I'm suggesting you cut 18 layers of complexity way down and build back up. Try building the PEX of the PexProcess by hand using the

pex

CLI. Mess with options, debug, get that working, build back up.

ahh I see. so build the pex to just run a jupyter notebook.

Yup

Once that works, see if I can run jupyter-book, etc.

Exactly

Thanks for the suggestions. I'll do some reading on pex first then

If you like prose you're in for disappointment. Best docs are CLI help right now. Some prose here: https://pex.readthedocs.io/en/v2.1.119/

I think I found the issue. I can get jupyter-book to run correctly when I set it up with

pex $(pip freeze) -c jupyter-book -o jb.pex

. My understanding is that this sets up the pex to use a console script. So rather than running

python -m jupyter-book

it just runs

jupyter-book

in the pex env. When I use

PexProcess

I write

main=ConsoleScript("jupyter-book")

, but in the sandbox that is created there is a pex directory rather than a binary file, and the kernel issue comes up. Any ideas on what the difference is?

No need to guess. Use

-ldebug --no-local-cache

- all Process arguments will be printed, including the one that builds that PEX. Then you can directly compare using the Pex CLI and hopefully repro / isolate the difference.