https://pantsbuild.org/ logo
#general
Title
# general
n

nice-florist-55958

07/21/2022, 5:53 PM
Possible unintended behavior: I noticed that the lockfile Pants generates (and hence the pip friendly lock pex generates) "normalizes" the distribution names (project_name field in the JSON) to lowercase (and maybe other normalizations, not sure). While probably doesn't matter at all to pants/pex, it did lead to an error in one of our related scripts that consumes the pex exported lockfile. Examples are
qPythhon -> qpython
,
Sphinx -> sphinx
, etc. I haven't confirmed, but it's almost as-if it converts the distribution name to the exposed module name (but that would be even more surprising!).
e

enough-analyst-54434

07/22/2022, 7:59 AM
This is intended. See: + https://peps.python.org/pep-0503/#normalized-names + https://github.com/pantsbuild/pex/blob/2b3bdee8cd99ffab0538a63fdd36de8de356f857/pex/pep_503.py#L25-L30 So, I think your scripting needs to be PEP-compliant too to avoid getting tripped up.
n

nice-florist-55958

07/22/2022, 12:20 PM
Thanks John, didn't realize the distros had to be normalized like that, only the modules they install (esp. because pip lists/freezes the distro with its non-normalized name). Now I see there is a reverse normalization that should be done in the scripting. I'll double check, but shouldn't pex have normalized typing_extensions to typing-extensions based on that rule then?
I put this in the wrong thread originally so moving back to here: “””Another interesting one: apparently you can use either
typing_extensions
or
typing-extensions
for the requirement. The former is the exposed module and the latter is the official name of the requirement, but
pip
will accept either and list
typing-extensions
as installed for both cases (the
_
version redirects to
-
on pypi.org). Pants will accept either too, but maintains whichever version you provided in the lockfile, which then conflicts with pip.”””
e

enough-analyst-54434

07/22/2022, 3:39 PM
I'll double check, but shouldn't pex have normalized typing_extensions to typing-extensions based on that rule then?
It should have. Checking just now (I tried Pex 2.1.90 & Pex 2.1.99) it appears to:
Copy code
$ pex3 lock create typing_extensions --style universal --resolver-version pip-2020-resolver | jq '.locked_resolves[] | .locked_requirements[]'
{
  "artifacts": [
    {
      "algorithm": "sha256",
      "hash": "25642c956049920a5aa49edcdd6ab1e06d7e5d467fc00e0506c44ac86fbfca02",
      "url": "<https://files.pythonhosted.org/packages/ed/d6/2afc375a8d55b8be879d6b4986d4f69f01115e795e36827fd3a40166028b/typing_extensions-4.3.0-py3-none-any.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "e6d2677a32f47fc7eb2795db1dd15c1f34eff616bcaf2cfb5e997f854fa1c4a6",
      "url": "<https://files.pythonhosted.org/packages/9e/1d/d128169ff58c501059330f1ad96ed62b79114a2eb30b8238af63a2e27f70/typing_extensions-4.3.0.tar.gz>"
    }
  ],
  "project_name": "typing-extensions",
  "requires_dists": [],
  "requires_python": ">=3.7",
  "version": "4.3"
}
Pants will accept either too, but maintains whichever version you provided in the lockfile, which then conflicts with pip.
Can you tighten that up for me? What exactly do you do and what exactly is the error / conflict you see with Pip? So, one thing to understand in case it's not already clear - the distribution / project name need have nothing to do with the top-level modules or top-level package names in the distribution. A familiar offender is the
setuptools
project. It does have a top-level
setuptools
package but it also has a top-level
pkg_resources
package. Closer to home, the `pantsbuild.pants`project only has a top-level `pants`package - not a match at all.
n

nice-florist-55958

07/22/2022, 6:01 PM
It could have just been cacheing or something and I wasn't seeing the update when tinkering around with it. I'll check back later. And yah, the distro name <> installed modules stuff I know pretty well now. :)
e

enough-analyst-54434

07/22/2022, 6:14 PM
Ok.
5 Views