https://pantsbuild.org/ logo
Join Slack
Powered by
trying to get `./pants test <single-file>` t...
# general
p
trying to get 
./pants test <single-file>
to work: Got errors that it can’t find 3rd party libraries like sqlalchemy, google.protobuf, factory (see inside)
Copy code
Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/v2.14/docs/troubleshooting#import-errors-and-missing-dependencies>.
16:10:48.37 [WARN] Pants cannot infer owners for the following imports in the target common/application/models.py:

  * sqlalchemy.Boolean (line: 8)
  * sqlalchemy.Column (line: 8)
  * sqlalchemy.DateTime (line: 8)
  * sqlalchemy.String (line: 8)
  * sqlalchemy.dialects.postgresql.JSON (line: 9)
  * sqlalchemy.dialects.postgresql.UUID (line: 9)
  * sqlalchemy.func (line: 8)

If you do not expect an import to be inferrable, add `# pants: no-infer-dep` to the import line. Otherwise, see <https://www.pantsbuild.org/v2.14/docs/troubleshooting#import-errors-and-missing-dependencies> for common problems.
16:10:48.37 [WARN] Pants cannot infer owners for the following imports in the target <>/proto/tfdv_pb2.py:

  * google.protobuf.descriptor (line: 6)
  * google.protobuf.internal.enum_type_wrapper (line: 5)
  * google.protobuf.message (line: 7)
  * google.protobuf.reflection (line: 8)
  * google.protobuf.symbol_database (line: 9)
Copy code
ImportError while loading conftest '/private/var/folders/xz/gwhmgmz14510pl52tgmhdl740000gn/T/pants-sandbox-YH9TnA/common/conftest.py'.
common/conftest.py:8: in <module>
    import factory
E   ModuleNotFoundError: No module named 'factory'
I ran 
./pants --filter-target-type=python_requirement list ::
but don’t see any 3rd party things. went through this doc, didn’t find anything. all my build files are up to date
b
Have you told Pants about your third party dependencies? Pants requires explicit specification of what third party dependencies you're planning to use (via 
python_requirement
or one of the generators, from a 
pyproject.toml
or 
requirements.txt
), and will then infer which files use them. https://www.pantsbuild.org/docs/python-third-party-dependencies has more docs
p
yes in common/BUILD I have:
Copy code
python_sources()
python_requirements(
    name="reqs",
    source="base-requirements.txt",
)

python_requirements(
    name="reqs0",
)
and the test is in common/foo/tests/test_foo.py I also took out the first base-requirements.txt and to make sure and still have the same error
I am in the failed sandbox 
pants-sandbox-lm1s6E
, saw some first party files. not sure how to check if any 3rd party dependencies are installed there
b
p
yes
not in sandbox, only in my main repo
Copy code
[python]
tailor_pex_binary_targets = false
enable_resolves = true
I ran
Copy code
./pants dependencies common/foo/tests/test_foo.py
I get:
Copy code
16:39:31.68 [INFO] Initializing scheduler...
16:39:31.95 [INFO] Scheduler initialized.
16:39:38.45 [WARN] The target common/foo/tests/test_foo.py:tests0 imports `grpc`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['…'].

Please explicitly include the dependency you want in the `dependencies` field of common/foo/tests/test_foo.py:tests0, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.

Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/v2.14/docs/troubleshooting#import-errors-and-missing-dependencies>.
so thats probably why..
b
That would suggest to me that dependencies like 
grpc
are being imported into pants multiple times? Are they listed in multiple requirements.txt files?
p
yes
it’s a monorepo so we have multiple requirements.txt
they all have the same version so the lockfile is able to be generated
b
I think pants' general expectation is that there's a single source of dependencies across all the whole repo. For example, in our pants repo, we have one 
backend/requirements.txt
that specifies all dependencies, and then subprojects like 
backend/api
and 
backend/whatever
that don't have their own requirements.txt. If you're migrating an existing codebase to pants, I think you can leave the individual-project 
requirements.txt
where they are without a 
python_requirements
(and use https://www.pantsbuild.org/docs/reference-python#tailor_requirements_targets to ensure 
./pants tailor ::
doesn't add them), and have a single top-level requirements.txt that is imported via 
python_requirements
. (This is what we did for our migration.) Alternatively, a path that seems to be harder would be to have a separate 'resolve' per sub-project (https://www.pantsbuild.org/docs/python-third-party-dependencies#multiple-lockfiles).
p
Thanks thats an awesome explanation/suggestion, appreciate it!
I ended up removing tailor_requirements_targets, resolve them to something other than default, and use pex3 to generate a single req.txt from the lockfile and use that as default
4 Views
Open in Slack
PreviousNext
Powered by