Hi there, I am currently evaluating to move to Pants for one of our Teams’ Multi/Monorepositories, where we currently use a poetry setup (with a lot of small scripts to make it work). It looks very promising so far and setting it up for automated tasks & artefact building was really easy. However, when I was looking into setting up VS Code (which most of us use) I am struggling a bit. Right now we install the needed first party packages es editable installs into multiple venvs (which correspond to different sets of lock files). There are multiple problems, for which I am not sure if there would be better approaches: • The code completion works best if all first party module paths are added as an extraPaths settings (which is read from a settings.json). For now I think it’s easiest if I use the output of

./pants roots

as every root in my setup corresponds to one top level module in the python module hierarchy. I guess this process can be easily automated and I assume there is no better solution here (and it’s not that bad with this setup, actually it’s better than what we had with poetry). • For multiple lock files, we can work around this for now and looking at the docs it seems there will be support for this soon, which is great 🎉! • Most headaches come from my tries to debug and run code from within VSCode. The setup with editable install works great with poetry as it easily allows to launch binaries / cli scripts from within VSCode with a debugger attached. As the files executed are the ones edited, there are no problems with source mapping and breakpoints set in VSCode are respected when running. For this VSCode uses debugpy (https://github.com/microsoft/debugpy). Executables are specified in a

launch.json

file (https://code.visualstudio.com/docs/python/debugging) and then it is possible to run them from the UI. ◦ I managed to debug a running pex file where I attach to a remote debugger manually started in the code. This is cool, but still a bit inconvenient as breakpoints are not working and it requires more boilerplate work. ◦ I also manage to run a pex file with the debugpy launcher, but then it ignores exceptions and does not enter the debugger (it seems some subprocess / fork is not covered by the debugger). ◦ Even with that I would need to do source mapping as the pex venv is located elsewhere and has some (possibly outdated) copies. ◦ One idea I had was a plugin that creates a venv where all source roots are installed editable. But not sure if there is an easier solution?

Hi there Malte! Thanks for that detailed feedback. As for your questions, I’ll defer them to the Pants veterans when they wake up 😉

Hi Andreas, thanks! I think we had a short chat on this topic on a github issue a while ago - but I didn’t manage to pick it up again until now 🙂

Hi @hallowed-student-25079 - I'm actually doing a bit of work on IDE support at the moment, so this is a good time to be asking about it.

Hi @happy-kitchen-89482, thanks - this is great to hear! Yes, this is when running under the run goal. We rarely use the vscode debugger on tests (as we execute them in the terminal usually - it might be something that could be nice if fully integrated at some point - but there we usually do some kind of embed of ptpython or ipython or pudb when debugging). Yesterday I spent an hour on hacking a plugin together that looks for

python_sources

targets in source roots (so top level modules) and creates a minimal setup.py (without requirements) in the directory, additionally the setup.py gets an entry point and requirements if there is a new

python_dev_module

target. My idea is now to also generate a script that can install a module at the top of the dependency chain (in our case that’s usually some kind of service with a an entrypoint) and also all the other modules that have no requirements into a venv. This way I can debug with vscode without any additional source mapping. This has the downside that the setup feels a bit hacky and is not exactly the same as with the run goal, as well as that I have to execute another command outside of pants (as I cannot execute the generated script from a goal rule). On the other hand, this is only needed whenever a top level module is added and the debug workflow is very much as vscode expect it to be setup.

Interesting! One thing I'd need to learn more about is the fact that you're consuming your own 1stparty code as editable installs, I don't quite understand that mode. What is the purpose of it?

What do you mean by consuming first party sources directly? I am not sure if this is possible with VSCode (might be and might also be an obvious solution) - but as vscode usually uses a shell + python interpreter to start debug processes the easiest way right now for us is to have a venv with interpreter and all code that will be loaded installed.

Yes, exactly

It can (https://code.visualstudio.com/docs/python/environments#_use-of-the-pythonpath-variable) and that might even be easier! It is just that most non mono repo setups I worked with used venvs with an editable install of the package itself (sometimes via poetry). Looking at the comment about the

.env

file - that one could probably be auto generated from the source roots and that would be easier (only entrypoints would not exist, but I think that’s not a problem). I will try this, thanks for the idea! Then depending on where my entrypoint module for debugging lies I only need to take care of having a venv with the right 3rd party dependencies. I guess for that one I could re-use the venv of a pex file right?

I too are used to having the sources in editable mode.. following this topic with keen eyes 🙂

For that reusing a venv part you may be interested in https://github.com/pantsbuild/pants/pull/13415

That looks exactly like what I would be wanting 🙂 . And it’s already merged, awesome! I will try it out once I have setup a pants from sources locally. I wanted to understand better how the PEX generation and execution works yesterday (before seeing your PR 🙂), so I created a

pex_binary

and had the execution mode set to

venv

. The binary has an entrypoint set and depends on a library (all dependencies inferred). This works when running

./pants run //:root_bin

but when I do

./pants package //:root_bin

and then

./dist/root_bin.pex

I get a

ModuleNotFoundError

and looking into the pex venv, I can see a directory for that module not found in the

site-packages

but no files in there. Interestingly the module some of the sources that are missing are depending on is also there (but empty too). Once I switch execution mode back to

zipapp

everything works fine. This is on 2.8.0rc1.

As it’s merged to main, you don’t have to do more than

PANTS_SHA=bb9f1991d5379747d435b3a997af715182bcaeab ./pants ...

to try it out… (using the latest git sha for main there…)

That’s very neat - I really like the whole UX/DX around pants 🙂. Also the problem I encountered on 2.8.0rc1 with the broken venv (the files itself were packed into the pex when unzipping) seems to be gone. I think I will continue my evaluation on the main branch. Will the current main branch changes be part of the 2.8.0 release?

I agree, big fan of how things are done in Pants here as well. 😄 Bugfixes are usually backported, but the major stuff on main now is for 2.9.x…

Yeah, this new ide support will not be in 2.8.0, but we cut new dev releases every week, so it'll be in the next one of those