Hi all :wave:! I'm new to Pants but excited to see...
# general
s
Hi all šŸ‘‹! I'm new to Pants but excited to see what it can do for migrating a multi-repo set up I have into a monorep. I'm hitting a
No module named main
error -- which I can see is due to having two projects with a
src/main.py
. Indicative project structure here:
Copy code
ā”œā”€ā”€ BUILD
ā”œā”€ā”€ apple-pie-api
ā”‚Ā Ā  ā”œā”€ā”€ BUILD
ā”‚Ā Ā  ā”œā”€ā”€ Pipfile
ā”‚Ā Ā  ā”œā”€ā”€ Pipfile.lock
ā”‚Ā Ā  ā”œā”€ā”€ requirements.txt
ā”‚Ā Ā  ā””ā”€ā”€ src
ā”‚Ā Ā      ā”œā”€ā”€ main.py
ā”‚Ā Ā          ...
ā”œā”€ā”€ common-requirements.txt
ā”œā”€ā”€ pants
ā”œā”€ā”€ pants.toml
ā””ā”€ā”€ randmoji-api
ā”‚   ā”œā”€ā”€ BUILD
ā”‚   ā”œā”€ā”€ Pipfile
ā”‚   ā”œā”€ā”€ Pipfile.lock
ā”‚   ā”œā”€ā”€ requirements.txt
ā”‚Ā Ā  ā””ā”€ā”€ src
ā”‚Ā Ā      ā”œā”€ā”€ main.py
ā”‚Ā Ā          ...
ā”œ   ...
I hit the issue when I try to execute the output pex for one of the projects; e.g., via
./pants package apple-pie-api:pex_binary && ./dist/apple-pie-api/pex_binary.pex
. Renaming one of the
main.py
(e.g., to
main2.py
) does the trick. But I'm wondering if there's a way to avoid needing to do that? Looking at the source roots https://www.pantsbuild.org/docs/source-roots this isn't in those example set ups, so perhaps I'm off piste here? But this is a set up I'd like to use as part of a transition to pants.
I have a rough-and-ready multi-project MWE here. (And any general always welcome šŸ™.)
e
Thanks for the repo. If you run the following you get a clue:
Copy code
$ ./pants package apple-pie-api:pex_binary
15:10:05.90 [WARN] The pex_binary target apple-pie-api:pex_binary has the field `entry_point='src/main.py'`, which maps to the Python module `main`, but Pants cannot safely infer a dependency because >1 target exports this module, so it is ambiguous which to use: ['apple-pie-api/src/main.py:../apple-pie-api', 'randmoji-api/src/main.py:../randmoji-api'].

Please explicitly include the dependency you want in the `dependencies` field of apple-pie-api:pex_binary, or ignore the ones you do not want by prefixing with `!` or `!!` so that <=1 targets are left.

Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target exports this module. Refer to <https://www.pantsbuild.org/v2.4/docs/troubleshooting#import-errors-and-missing-dependencies>.
15:10:05.91 [INFO] Wrote dist/apple-pie-api/pex_binary.pex
Which is followed via:
Copy code
$ git diff
diff --git a/apple-pie-api/BUILD b/apple-pie-api/BUILD
index 3f3f778..44e6ed6 100644
--- a/apple-pie-api/BUILD
+++ b/apple-pie-api/BUILD
@@ -7,6 +7,9 @@ python_library(
 pex_binary(
     name="pex_binary",
     entry_point="src/main.py",
+    dependencies=[
+        "apple-pie-api",
+    ]
 )
 
 # You need `python_requirements` to teach pants that this project has third-party requirements
Which nets you:
Copy code
$ ./pants package apple-pie-api:pex_binary
15:10:00.12 [INFO] Wrote dist/apple-pie-api/pex_binary.pex
$ zipinfo dist/apple-pie-api/pex_binary.pex | tail -10
-rw-r--r--  2.0 unx      558 b- defN 80-Jan-01 00:00 .bootstrap/pex/vendor/_vendored/setuptools/pkg_resources/py31compat.py
-rw-r--r--  2.0 unx      984 b- defN 80-Jan-01 00:00 .bootstrap/pex/venv_bin_path.py
-rw-r--r--  2.0 unx      155 b- defN 80-Jan-01 00:00 .bootstrap/pex/version.py
-rw-r--r--  2.0 unx      590 b- defN 80-Jan-01 00:00 PEX-INFO
-rw-r--r--  2.0 unx     2868 b- defN 80-Jan-01 00:00 __main__.py
-rw-r--r--  2.0 unx      932 b- defN 80-Jan-01 00:00 main.py
drwxr-xr-x  2.0 unx        0 b- stor 80-Jan-01 00:00 numerics/
-rw-r--r--  2.0 unx       55 b- defN 80-Jan-01 00:00 numerics/__init__.py
-rw-r--r--  2.0 unx      658 b- defN 80-Jan-01 00:00 numerics/pi.py
And the desired outcome:
Copy code
$ ./dist/apple-pie-api/pex_binary.pex 
Traceback (most recent call last):
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/.bootstrap/pex/pex.py", line 489, in execute
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/.bootstrap/pex/pex.py", line 406, in _wrap_coverage
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/.bootstrap/pex/pex.py", line 437, in _wrap_profiling
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/.bootstrap/pex/pex.py", line 545, in _execute
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/.bootstrap/pex/pex.py", line 672, in execute_entry
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/.bootstrap/pex/pex.py", line 684, in execute_module
  File "/usr/lib/python3.8/runpy.py", line 210, in run_module
    return _run_code(code, {}, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/jsirois/dev/mattjw/jsirois-pants-multiproject-mwe/dist/apple-pie-api/pex_binary.pex/main.py", line 6, in <module>
ModuleNotFoundError: No module named 'uvicorn'
Well - not quite the desired outcome, but now you're on the road to working more things out.
The short of it is this: If you must introduce duplicate names, you'll be foiling Pants dependency inference and you'll need to disambiguate for every duplicate name you introduce with a manual explicit dependency added to a BUILD file.
šŸ‘ 1
Does that make sense?
Another way to view this is - "No it doesn't make sense. Of course I mean the src/main.py thats right here where I said!". And you're right. But Pants isn't that smart yet. It only deals in calculated module names, and only after calculating those, which it does without regard to locality, does it see there are >1
main
modules.
s
Thanks @enough-analyst-54434! Just digesting now!
šŸ’” Thank you! It works. I'll explore more about the why ā€“ your explanation is more than enough for me to get started on that. I'll have a look at
ModuleNotFoundError: No module named 'uvicorn'
too. I didn't hit it on my local, so maybe the requirements.txt got re-generated on yours? Which is indeed what my (misleading) instructions do suggest to do šŸ˜….
Now that the ambiguous
main.py
is fixed I'm re-thinking my assumptions about mixing requirements files (and common requireemnts). I'll come back here if i hit any other issues. Thanks again!
šŸ‘Œ 2