https://pantsbuild.org/ logo
Join Slack
Powered by
in python we're having a hard time with dependency...
# general
c
in python we're having a hard time with dependency inference. it looks like everything is set up correctly but relying on 
pants run
instead of poetry it just can't find the first-party imports and bombs out on no module found errors, It isn't stuff we should need to declare explicitly but tried that anyway. anybody else have similar issues? It's working fine for imports from a 'shared' directory with logging config in it that's coming into the app, and because that's at the top level it's declared in source roots. so I'm wondering if it's something related to that.
e
It's almost certainly source roots. You want to be able to run 
pants roots
and confirm the printed list includes the start of all your import paths; ie: a valid 
PYTHONPATH
for your source tree. Definitely slow down and get that right 1st. Almost nothing else should be attempted before understanding source roots and configuring them correctly.
c
best answer, thank you. I was feeling much the same.
for future travelers: after reviewing that again in detail we got most of this working, seems like the issue is that we assumed if we had a source root like 
/
, 
src/python
, etc. we could import something like 
import src.python.whatever
wherever we needed to throughout the codebase. I think because we had a lower-level source root 
src/python
what we needed to do is have our import statement pick up where the source root left off a la 
import .whatever
in our case because we were trying to import from the directory above the containing file. and that directory was immediately below 
src/python
this seems to be working as long as the imports aren't coming from within an 
__init__.py
file. pants doesn't like that much I'm guessing because of the way python import syntax handles it as just the module name. we seemed to have worked around that by creating a resource target for that 
__init__.py
file within the same directory as the file, setting that file as a source. I'm only mostly sure about my understanding of why it's working or not in various scenarios and I guess that's the journey, but maybe this helps somebody
e
You still sound a bit iffy on your understanding. I'd really nail it down if I were you. All it really should boil down to is, from a CWD of the repo root, with a venv activated that has all your 3rdparty deps, get 
PYTHONPATH=X python -c 'import my.app.module'
to work. Whatever the contents of X you have for 
PYTHONPATH
to have gotten that working - that's what the Pants source roots need to be for Python.
Note in my test rig suggestion there is no Pants in sight. It's just Python and how Python works. You definitely want to understand that. It is ground truth here.
c
exactly! and for example, if I have 
/
in 
source_roots
(specifying the monorepo root dir) presumably I would hope that puts it in PYTHONPATH, and I would expect to be able to use imports throughout my app with no issue provided the dependencies I'm importing are under that point in the directory structure (provided I start my import statement from where that source_root path leaves off.) I believe to have observed that this understanding is not correct, though I'm going to go back and test it now. maybe I just got turned around. or maybe it didn't make it onto PYTHONPATH (I'm fairly sure I checked.)
also 
pants run
seems to always run from the monorepo root so for applications like django that expect things to be in certain places relative to itself it's been a bit of a tug of war
worked around some of it with explicit sys.path statements adding things to PYTHONPATH in 
manage.py
and in my pytest configuration (to handle 
pants test
scenarios that don't run via manage.py)
e
exactly! and for example, if I have / in source_roots (specifying the monorepo root dir) presumably I would hope that puts it in PYTHONPATH, and I would expect to be able to use imports throughout my app with no issue
But, you said you have Python code under 
src/python
. Surely 
src.python
is not a Python package. Surely the packages start under 
src/python
, no?
@careful-airline-93122 you should almost certainly have 
src/python
listed as a source root if this repo is setup using words in the way they typically are used. Having just 
/
as a source root is almost positively incorrect unless you also have bare modules in the root. Do you? Like do you also have 
example.py
in the repo root that should be imported as 
import example
in addition to modules under 
src/python
like 
src/python/foo/bar.py
that should be imported as 
from foo import bar
?
I'm almost positive you still don't fully understand how plain old Python - no Pants - works wrt CWD and PYTHONPATH and imports.
If the confusion continues and you can provide an example repository that is public, I can help debug that.
c
what you just said clicked; apologies for my confusion. I think we got the folder structure from the docs; we don't have .py files at the root, but we have shared resources in another root module 
shared/python
which I'm realizing based on what you said would be just another source root and no different from any of the other modules. so having 
/
only as a source root would not make sense as it's not a python module, but a directory containing them. hopefully I'm grokking that right, and I appreciate you taking the time to explain.
ah. I must have read right past this several times.
e
When I was a kid there was a bookmobile program nationwide? where a physical school bus would come to your school full of books. The TV ads for this centered around R.I.F: reading is fundamental Boy is it. To this day I do what you did too often and read too quickly/ poorly.
c
slowly and poorly, 10th times a charm
e
Here it is!:

https://youtu.be/6x4SGvjErZA?si=dZ1fzAgI3ZlX9ot_▾

😂 1
2 Views
Open in Slack
PreviousNext
Powered by