Possible unintended behavior: I noticed that the lockfile Pants generates (and hence the pip friendly lock pex generates) "normalizes" the distribution names (project_name field in the JSON) to lowercase (and maybe other normalizations, not sure). While probably doesn't matter at all to pants/pex, it did lead to an error in one of our related scripts that consumes the pex exported lockfile. Examples are

qPythhon -> qpython

,

Sphinx -> sphinx

, etc. I haven't confirmed, but it's almost as-if it converts the distribution name to the exposed module name (but that would be even more surprising!).

This is intended. See: + https://peps.python.org/pep-0503/#normalized-names + https://github.com/pantsbuild/pex/blob/2b3bdee8cd99ffab0538a63fdd36de8de356f857/pex/pep_503.py#L25-L30 So, I think your scripting needs to be PEP-compliant too to avoid getting tripped up.

Thanks John, didn't realize the distros had to be normalized like that, only the modules they install (esp. because pip lists/freezes the distro with its non-normalized name). Now I see there is a reverse normalization that should be done in the scripting. I'll double check, but shouldn't pex have normalized typing_extensions to typing-extensions based on that rule then?

I'll double check, but shouldn't pex have normalized typing_extensions to typing-extensions based on that rule then?

It should have. Checking just now (I tried Pex 2.1.90 & Pex 2.1.99) it appears to:

$ pex3 lock create typing_extensions --style universal --resolver-version pip-2020-resolver | jq '.locked_resolves[] | .locked_requirements[]'
{
  "artifacts": [
    {
      "algorithm": "sha256",
      "hash": "25642c956049920a5aa49edcdd6ab1e06d7e5d467fc00e0506c44ac86fbfca02",
      "url": "<https://files.pythonhosted.org/packages/ed/d6/2afc375a8d55b8be879d6b4986d4f69f01115e795e36827fd3a40166028b/typing_extensions-4.3.0-py3-none-any.whl>"
    },
    {
      "algorithm": "sha256",
      "hash": "e6d2677a32f47fc7eb2795db1dd15c1f34eff616bcaf2cfb5e997f854fa1c4a6",
      "url": "<https://files.pythonhosted.org/packages/9e/1d/d128169ff58c501059330f1ad96ed62b79114a2eb30b8238af63a2e27f70/typing_extensions-4.3.0.tar.gz>"
    }
  ],
  "project_name": "typing-extensions",
  "requires_dists": [],
  "requires_python": ">=3.7",
  "version": "4.3"
}

Pants will accept either too, but maintains whichever version you provided in the lockfile, which then conflicts with pip.

Can you tighten that up for me? What exactly do you do and what exactly is the error / conflict you see with Pip? So, one thing to understand in case it's not already clear - the distribution / project name need have nothing to do with the top-level modules or top-level package names in the distribution. A familiar offender is the

setuptools

project. It does have a top-level

setuptools

package but it also has a top-level

pkg_resources

package. Closer to home, the `pantsbuild.pants`project only has a top-level `pants`package - not a match at all.

It could have just been cacheing or something and I wasn't seeing the update when tinkering around with it. I'll check back later. And yah, the distro name <> installed modules stuff I know pretty well now. :)

Ok.