https://pantsbuild.org/ logo
#general
Title
# general
a

adamant-coat-28831

09/01/2022, 7:25 PM
Hi, I have a question about imports.
☝️ 1
We are building a monorepo that will house our machine learning models. Broadly speaking, each model is contained in a directory that has a BUILD file. The setup looks something like this, where in this example there are 2 models: model_1 and model_2:
Copy code
monorepo/
	cicd/
		BUILD
		...
	libraries/
		BUILD
		...
	models/
		model_1/
			BUILD
			main.py
			model.py

		model_2/
			BUILD
			main.py
			model.py
	...
	pants
	pants.toml
The code in
main.py
is identical across
model_1
and
model_2
. We started with
model_1
and everything ran fine, but when we added model_2 and ran
./pants check ::
we obtained this warning:
Copy code
The target models/model_1/main.py:lib imports `<http://model.bm|model.bm>`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['models/model_2/model.py:lib', 'models/model_1/model.py:lib'].

Please explicitly include the dependency you want in the `dependencies` field of models/model_1/main.py:lib, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.

Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/v2.12/docs/troubleshooting#import-errors-and-missing-dependencies>.
we were able to remove the warning by changing the imports to relative imports, eg from
from model import bm
to
from .model import bm
.
There was also an error:
Copy code
models/model_1/main.py: error: Duplicate module named "main" (also at "models/model_2/main.py")
models/model_1/main.py: note: Are you missing an __init__.py? Alternatively, consider using --exclude to avoid checking one of them
that we addressed by adding
__init__.py
scripts to model_1 and model_2.
Is this the correct way to handle these imports and dependencies between modules?
b

bitter-ability-32190

09/02/2022, 2:58 PM
Usually yeah.
mypy
doesn't know you aren't using implicit namespace packages so it warns.
r

rich-london-74860

09/02/2022, 3:48 PM
But that looks like a pants error, not a mypy error?
What makes this problem worse for us is the fact that we’re deploying these individual models (e.g.
models/model_1
using MLFlow Project, which does not seem to handle relative imports. This means that we cannot use relative imports in MLFlow Projects, where the code actually runs, and we can’t use implied imports in Pants. Does that means we just can’t use MLFlow Project and Pants?
b

bitter-ability-32190

09/02/2022, 4:34 PM
Whats your source roots?
r

rich-london-74860

09/02/2022, 4:35 PM
You mean as defined in
BUILD
?
b

bitter-ability-32190

09/02/2022, 4:35 PM
in
pants.toml
r

rich-london-74860

09/02/2022, 4:36 PM
We’re using `marker_filenames`M for our
Copy code
marker_filenames = ["setup.cfg", "MLProject"]
setup.cfg
to identify our libraries
MLProject
to identify our MLFlow Projects
b

bitter-ability-32190

09/02/2022, 4:37 PM
Is this Pants v2?
a

adamant-coat-28831

09/02/2022, 4:37 PM
2.12
b

bitter-ability-32190

09/02/2022, 4:38 PM
I suspect you might wanna bump your source roots a dir higher. Then you can import as
from model_1.model ...
☝️ 1