https://pantsbuild.org/ logo
Join Slack
Powered by
has anybody else seen this? if I add this to the c...
# general
i
has anybody else seen this? if I add this to the command line it works 
pants test --python-infer-ambiguity-resolution=by_source_root src/some/path
but this doesn't in pants.toml
Copy code
[python-infer]
# resolve the requirements.txt closest to the source root
ambiguity_resolution = "by_source_root"
f
You can see the state of what Pants thinks the configuration is, by running (in this case): 
pants help-advanced python-infer
For the 
ambiguity_resolution
config, you should see the current value displayed in the help output.
What does it display?
i
Copy code
--python-infer-ambiguity-resolution=<AmbiguityResolution>
  PANTS_PYTHON_INFER_AMBIGUITY_RESOLUTION
  ambiguity_resolution
      one of: [none, by_source_root]
      default: none
      current value: by_source_root (pants.toml)
f
So it appears to be set the way you want it then.
What issue are you seeing then?
i
it doesn't work when it is only set in the pants.toml file. it works when it is on the command line
I have multiple requirements.txt and this is just what I need
f
Are you able to share a full log of a failing Pants run?
i
while trying different targets, I'm getting different results where sometimes it associates the correct requirements.txt uniquely, and sometimes I get the warnings like (names changed)
Copy code
[WARN] The target beam/package/svc_x/entry_point.py imports `google.cloud.pubsub_v1`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['beam/package/svc_x2:reqs#google-cloud-pubsub', 'beam/package/svc_x3:reqs#google-cloud-pubsub', 'beam/package/svc_x4:reqs#google-cloud-pubsub', 'beam/package/svc_x5:reqs#google-cloud-pubsub', 'beam/package/svc_x6:reqs#google-cloud-pubsub', 'beam/package/svc_x7:reqs#google-cloud-pubsub', 'beam/package/svc_x8:reqs#google-cloud-pubsub', 'beam/package/svc_x9:reqs#google-cloud-pubsub', 'beam/package/svc_x10:reqs#google-cloud-pubsub', 'beam/svc_x11:reqs#google-cloud-pubsub', 'beam/svc_x12:reqs#google-cloud-pubsub', 'beam:reqs#google-cloud-pubsub'].
this morning experimenting with different test targets I'm getting those sorts of errors with or without a command line option. I've never seen the warning go away with the pants.toml configuration option. I will try to get to a more repeatable scenario. Could caching mean that the warnings went away for some other reason between runs?
since the ambiguity flag isn't working for me, the other suggestion is to update the dependencies field to exclude the ambiguous ones. Annoying but ok. but which dependencies field? python_sources like this? it doessn't seem to work, I get the same ambiguity warnings
Copy code
python_sources(
    dependencies=[
        '!beam/package/wedge_loads:reqs',       '!beam/package/file_upload_ingestor:reqs']
)

python_requirements(
    name="reqs",
    source="requirements_svc1_service.txt",
)
when I say "the other suggestion" I am referring to this in the warning
Copy code
Please explicitly include the dependency you want in the `dependencies` field of beam/svc_1/tests/package/end_2_end/tl_end_2_end_test.py:../../svc_1-tests, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.
f
And is the target shared library code or code foe one of the services?
And are you using the resolves feature for the different Python requirements?
i
service
not using resolves since we use pip-compile separately
my assumption there is that our requirements.txt variants are our lockfiles
f
That may be part of the problem then. You have these multiple requirements targets all in what is essentially a default resolve, thus Pants does not know which one to use since they are all "visible" to the source target.
Your use case is a non-standard use case. An ordinary Pants solution would be to either use multiple resolves or to just merge the requirements into a single resolve and then allow Pex to slice the lockfile as necessary to only use the dependencies actually needed by particular sources (i.e., your services).
i
ok, I'll pick my poison! truthfully today, the separate requirements files are only applied to the docker container, the CI/CD system using the merged requirements.txt for everything. And because of that we've missed some imports that then failed at runtime after deployment. So my hope was to be able to run tests with the actual requirements so that problem would be caught much sooner
thanks for the help
2 Views
Open in Slack
PreviousNext
Powered by