https://pantsbuild.org/ logo
#general
Title
# general
c

curved-manchester-66006

12/21/2022, 4:21 AM
For "fun" dependency resolution reasons (thanks for pipping
pip_version
thru!) this is on
2.16.0.dev3
so feel free to defer. My question is basically a gut check on "this looks like source roots or other common onboarding confusion" or "probably something with the dev release, check back in January". I'm getting variants of this error message:
Copy code
./pants run scripts/hello.py 
Traceback (most recent call last):
...
ImportError: No module named scripts.hello
🧵
1
Copy code
$ ./pants roots
.
$ cat scripts/hello.py 
import modules.critical

print('hello')
$ cat modules/hello.py 
import modules.critical

print('hello')
$ ./pants run modules/hello.py 
hello
$ ./pants run modules/critical.py
$ ./pants run scripts/hello.py 
Traceback (most recent call last):
  File "/tmp/pants-sandbox-uOaYuI/./.cache/pex_root/venvs/27fadc29d37cd72af28911c9a5a9bb68222b5d90/4f398b5364288224a985fd546e447fdd048b6796/pex", line 248, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/usr/local/lib/python3.7/runpy.py", line 201, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/usr/local/lib/python3.7/runpy.py", line 136, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named scripts.hello
Where
modules.critical
is a real class with various 1st and 3rd party dependencies. However:
Copy code
$ cat modules/bar.py 
bar='bar'
$ cat scripts/hello2.py 
import modules.bar

print('hello')
$ ./pants run scripts/hello2.py 
hello
I have not been able to find a repo case between the trivial example and the real code that blows up.
c

curved-television-6568

12/21/2022, 3:15 PM
strange (thanks for trying out the dev releases, that helps find issues earlier!)
couple of Q’s that comes to mind: if you comment out the
import modules.critical
in
scripts/hello.py
does it work then? do you miss any
__init__.py
files? does the
./pants dependencies --transitive scripts/hello.py
look sane? if you run with
--keep-sandboxes=always
and inspect/run the
__run.sh
from there (also check what files you have in the sandbox may be instructive)
c

curved-manchester-66006

12/21/2022, 3:54 PM
f you comment out the
import modules.critical
in
scripts/hello.py
does it work then?
Yes.
Copy code
$ cat scripts/hello.py 
#import modules.critical

print('hello')
$ ./pants run scripts/hello.py 
hello
do you miss any
__init__.py
files?
Copy code
$ ls modules/__init__.py 
modules/__init__.py
$ ls scripts/__init__.py 
scripts/__init__.py
I get the same behavior with and without a
__init__.py
at the repo root (eg
$ touch __init__.py
)
does the
./pants dependencies --transitive scripts/hello.py
look sane?
It's not obviously wrong to me, but it is also > 100 lines long. Like I said I haven't been able to get this to a minimal reproduction.
if you run with
--keep-sandboxes=always
and inspect/run the
__run.sh
from there (also check what files you have in the sandbox may be instructive)
I'd totally believe this is something silly with
__init__..py
files or whatnot, but I'm still not seeing it. It seems to work when I run it interactively.
Copy code
/tmp/pants-sandbox-jxnlqA $ ./scripts.pex_pex_shim.sh
Traceback (most recent call last):
  File "/tmp/pants-sandbox-jxnlqA/.cache/pex_root/venvs/27fadc29d37cd72af28911c9a5a9bb68222b5d90/4f398b5364288224a985fd546e447fdd048b6796/pex", line 248, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/usr/local/lib/python3.7/runpy.py", line 201, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/usr/local/lib/python3.7/runpy.py", line 136, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named scripts.hello
Copy code
/tmp/pants-sandbox-jxnlqA $ ./scripts.pex_bin_python_shim.sh
Python 3.7.15 (default, Dec 19 2022, 16:05:03) 
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import scripts.hello
/tmp/pants-sandbox-jxnlqA/.cache/pex_root/venvs/27fadc29d37cd72af28911c9a5a9bb68222b5d90/4f398b5364288224a985fd546e447fdd048b6796/lib/python3.7/site-packages/airflow/configuration.py:276: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if StrictVersion(sqlite3.sqlite_version) < StrictVersion(min_sqlite_version):
hello
>>>
(Thanks for taking a look!)
👍 1
c

curved-television-6568

12/21/2022, 4:39 PM
the sandbox scenario seems to be worth exploring, as you’ve got a working and a failing scenario from there.. I’d look at what the sys.path and relevant env vars are for each of those.. 🙂
c

curved-manchester-66006

12/21/2022, 9:34 PM
Thank you for your suggestion regarding the sandbox! Although I think I've now found a python packaging footgun and not a pants bug. The
sys.path
in the sanbox looks (from my dumb print statement debugging) to be:
['/home/ecsb/.cache/pants/named_caches/pex_root/venvs/27fadc29d37cd72af28911c9a5a9bb68222b5d90/4f398b5364288224a985fd546e447fdd048b6796', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/home/ecsb/.cache/pants/named_caches/pex_root/venvs/s/19a57b50/venv/lib/python3.7/site-packages']
Inside the first part there is.... a lot of stuff
Copy code
ls -1 /home/ecsb/.cache/pants/named_caches/pex_root/venvs/27fadc29d37cd72af28911c9a5a9bb68222b5d90/4f398b5364288224a985fd546e447fdd048b6796/lib/python3.7/site-packages/ | wc -l
Notable:
Copy code
ls  /home/ecsb/.cache/pants/named_caches/pex_root/venvs/27fadc29d37cd72af28911c9a5a9bb68222b5d90/4f398b5364288224a985fd546e447fdd048b6796/lib/python3.7/site-packages/scripts/
build_api_docs.py            check_for_technical_terms.py  __init__.py         validate_docs_snippet_line_numbers.py
build_glossary_page.py       check_type_hint_coverage.py   __pycache__
check_docstring_coverage.py  docs_link_checker.py          trace_docs_deps.py
^^^ Some transitive dep which also has a top level
scripts
module!!! Back to my original example, if I do
Copy code
$ cp -ra scripts crazy_scripts
$ ./pants   run crazy_scripts/hello.py 
16:32:25.50 [INFO] Initializing scheduler...
16:32:26.10 [INFO] Scheduler initialized.
hello
It works! I guess to tie it back to pants, this is something it could maybe warn about?
c

curved-television-6568

12/21/2022, 10:11 PM
AH! yeah, that would’ve been a nice service for sure 👍
3 Views