hey again it seems I ran into an issue where some 3rd party Pants #general

Join Slack

hey again. it seems I ran into an issue where some...

# general

freezing-fall-2672

09/14/2023, 8:57 AM

hey again. it seems I ran into an issue where some 3rd party python packages are found with a (implicite) mapping of

with

and some are not. strarting from django settings I have f.e.

Copy code

'django_countries',
'django_json_widget',

and in requirements.txt

Copy code

django-countries==7.5
django-json-widget==1.1.1

so far so good, but I run into

ModuleNotFoundError: No module named 'django_json_widget'

when trying to run a test. Any ideas about this?

freezing-fall-2672

09/14/2023, 9:00 AM

I tried to fix this with an module_mapping in the python_requirements target.

late-advantage-75311

09/14/2023, 9:02 AM

There are a bunch of built-in mappings in https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/dependency_inference/default_module_mapping.py that are generally applied for known packages. Your two are not listed there in the django section, so that is why you are having to do it yourself

late-advantage-75311

09/14/2023, 9:02 AM

And it isn't working?

late-advantage-75311

09/14/2023, 9:05 AM

If you are going off of https://www.pantsbuild.org/docs/python-third-party-dependencies#use-modules-and-module_mapping-when-the-module-name-is-not-standard and it isn't working, def post more details about the example (pants.toml, BUILD and the consuming file) and we can try to see together what is going on.

freezing-fall-2672

09/14/2023, 9:21 AM

I'm not sure if default module mapping is involved here since it's working for

django-countries

(which is loaded first and not in the defaults) and not working for

django-json-widget

. so my guess would be the translation of pypi package name with a dash to an underscore in the source package name - but that can't be right since it doesn't work ... 🤔

freezing-fall-2672

09/14/2023, 9:22 AM

and yes, I'm trying to solve it with 'Use

modules

and

module_mapping

when the module name is not standard' but somehow it doesn't work - for this example. For other modules like f.e.

"django-autocomplete-light": ["dal"],

it's working just fine.

late-advantage-75311

09/14/2023, 9:24 AM

oh you're probably right, that the default is converting

between

and only then applying those hardcoded mappings

freezing-fall-2672

09/14/2023, 9:25 AM

somehow though f.e. for

"django-admin-rangefilter": ["rangefilter"],

it seems to not work

late-advantage-75311

09/14/2023, 9:26 AM

btw, how are the imports set up? it is in a django settings file rather than a direct import?

freezing-fall-2672

09/14/2023, 9:26 AM

here it's about the settings file

late-advantage-75311

09/14/2023, 9:27 AM

if you run

pants dependencies your-source-target

what do you see?

late-advantage-75311

09/14/2023, 9:28 AM

your-source-target being the address of the source file that you expect depends on the particular requirement

freezing-fall-2672

09/14/2023, 9:28 AM

without the transitive flag I don't see f.e. dal

freezing-fall-2672

09/14/2023, 9:28 AM

with the flag it's in

foo/bar:reqs#django-autocomplete-light

freezing-fall-2672

09/14/2023, 9:29 AM

(needed to edit because of names 😆 )

freezing-fall-2672

09/14/2023, 9:30 AM

aaah

freezing-fall-2672

09/14/2023, 9:30 AM

so, I have a mismatch between the actual dependencies from the source target and what I try to import from settings

late-advantage-75311

09/14/2023, 9:32 AM

Ok, if you have a lead go ahead. If you still get stuck, you may have to provide some more details (ideally an example repo) for us to minimally reproduce the issue and I would be happy to try to understand what is going on. Until then I think I am too new to provide more useful suggestions.

freezing-fall-2672

09/14/2023, 9:32 AM

yeah I can't really provide this as an example since it's closed source 🤷

freezing-fall-2672

09/14/2023, 9:33 AM

anyways, you already helped me - thx!

late-advantage-75311

09/14/2023, 9:34 AM

you can always adapt the example to a minimal open source repo that reproduces the issue (like with

foo.py

instead of

some_trade_secret_stuff.py

) It is a common technique used on this channel and is very effective for getting help and increasing collective knowledge.

freezing-fall-2672

09/14/2023, 9:38 AM

sure, already did this https://github.com/elmcrest/example-django-multiple-monoliths-pantsbuild

freezing-fall-2672

09/14/2023, 9:39 AM

here though it's the actual large code base and not really easy to reproduce, just saying ... of course as the last resort that's probably what I'd have to do

freezing-fall-2672

09/14/2023, 9:39 AM

I try to avoid it though - feels like weeks of work 😆

late-advantage-75311

09/14/2023, 9:46 AM

It may be a false economy, Marius. In constructing that minimal example you may end up solving it - the smallest repo that can reproduce the issue you are seeing. (I'm sure you've had that experience lots of times in regular development), or you may find that what you are experiencing as an issue is actually two distinct issues. Anyway, that's my bit of encouragement for the day 🙂

freezing-fall-2672

09/14/2023, 9:52 AM

😄 🤝

2 Views

Open in Slack

Previous Next