Hi, is there some standard way of handling python ...
# general
r
Hi, is there some standard way of handling python sub-packages which aren't necessarily imported inside but the package under which these sub-packages live. These sub-packages are just a way to modularize the code base. e.g.
Copy code
package
  BUILD
  __init__.py
  -- sub-pkg1
  -- sub-pkg2
then later we do say for testing
Copy code
from package.sub-pkg1 import func1
pants will complain that this
package.sub-pkg1
isn't a package since it can't infer the dependency unless I explicitly add in BUILD
sub-pkg1
as dependency for
package
. Basically is there a way around providing this explicit dependency in such case.
1
f
what happens if you create
sub-pkg1/__init__.py
?
r
it still complains
I did some
./pants dependencies package:package
and it doesn't include
sub-pkg1
as dependency unless I add it explicitly
f
and what are the contents of
package/BUILD
and
package/sub-pkg1/BUILD
?
w
if you ever need to manually add a dependency that you have a Python import statement for, it’s most likely due to not having source roots set up properly: https://www.pantsbuild.org/docs/source-roots
f
I didn’t see
package/sub-pkg1/BUILD
listed as a file. If that is missing, you will need a
python_sources
target in that BUILD file.
./pants tailor
can make all of the necessary
BUILD
files for you.
r
The import statement is not inside the
package
though. That's why pants can't infer it. This is just a way to organize code inside
package
@fast-nail-55400 yeah all the
BUILD
inside
sub-pkg
have
python_sources
f
where is the import statement located then?
r
structure of repo is something like this
Copy code
package1
package2
tests
and then when I run
./pants test tests:tests
for testing
package1
and
package2
, imports inside tests are like
from package1.sub-pkg1 import func_to_test
which complains
f
then @witty-crayon-22786’s suggestion about checking the source root configuration in pants.toml probably applies
you will likely need to list the root of the repo as a source root
h
you can run
./pants roots
r
I keep running into this issue where I need to explicitly add dependency for my tests import to work if sub packages etc weren't imported inside the packages where these sub packages live. I verified source root etc. This work out of the box if I am directly running pytest though. It's able to figure out these sub packages.
I'll try to reproduce and create an example repo
👍 1
So this seems to happen because these sub packages were imported inside
___init___.py
of
tests
. I moved all such imports to
conftest.py
where I define some pytest fixtures. This fixes the issue. Not sure if this is how it should be i.e. using conftest/fixture. It does make more sense. On the other hand pytest on its own doesn't complain when importing everything inside
___init___.py
Even in this case pytest runs the test successfully.
w
ah. by default, Pants does not infer dependencies on
__init__.py
files: you can enable it with: https://www.pantsbuild.org/docs/reference-python-infer#section-inits