https://pantsbuild.org/ logo
Join Slack
Powered by
Hi everyone! :wave: I'm looking for some help. I'm...
# general
m
Hi everyone! 👋 I'm looking for some help. I'm in the process of upgrading python and pants versions on some old code to install a private auth package (requires python 3.11). My current issue is getting this error when attempting to run a test that imports the auth package:
Copy code
ERROR: Could not find a version that satisfies the requirement endpoint-authenticator==1.1.5 (from versions: none)
ERROR: No matching distribution found for endpoint-authenticator==1.1.5
When running in debug mode I can see :
Copy code
[DEBUG] Completed: Find interpreter for constraints: CPython>=3.11
and
Copy code
[DEBUG] Running Building requirements.pex with 10 requirements: aws-lambda-powertools==1.31.1, boto3==1.26.62, endpoint-authenticator==1.1.5, nanoid==2.0.0, psycopg2-binary==2.9.5, pytest-mock==3.10.0, pytest==7.2.1, requests-mock==1.10.0, requests==2.28.2, sqlalchemy==1.4.46 under semaphore with concurrency id: 2, and concurrency: 10
which leads me to believe that it can see 
endpoint-authenticator
and has the correct interpreter. Disclaimer: I am a python and pants newb 🙂 Would appreciate any help y'all can provide!
pants.toml file:
Copy code
[GLOBAL]
pants_version = "2.17.1"
backend_packages = [
  "pants.backend.python",
  "pants.backend.awslambda.python",
]

[source]
# The Python source root is the repo root. See <https://www.pantsbuild.org/docs/source-roots>.
root_patterns = ["/"]

[python-bootstrap]
search_path = [
    # This will use all interpreters in `$(pyenv root)/versions`.
    "<PYENV>",
     # Location on the Codefresh docker build. 
    "/usr/local/bin/",
    # Location for WSL
    "/usr/bin/",
]

[test]
use_coverage = true

[pytest]
version = "pytest>=7.0.1"
extra_requirements.add = [
  "pytest-env>=0.6.2",
]
lockfile = "pantsPytest.lock"

[coverage-py]
fail_under = 80

[anonymous-telemetry]
enabled = false

[python]
interpreter_constraints = [">=3.11"]

[python-repos]
indexes.add = ["http://%(env.JFROG_USERNAME)s:%(env.JFROG_ACCESS_TOKEN)s@endpoint.jfrog.io/artifactory/api/pypi/pypi/simple/index"]
I was also getting this error:
Copy code
from endpoint.authenticator.auth_service import AuthService
ModuleNotFoundError: No module named 'endpoint'
Which (I believe) I resolved by adding this to my 
BUILD
file
Copy code
python_requirement(
    name="endpoint-authenticator",
    requirements=["endpoint-authenticator==1.1.5"],
    modules=["endpoint.authenticator"],
)
The auth package exists in both my 
poetry.lock
and 
pyproject.toml
respectively as
Copy code
[[package]]
name = "endpoint-authenticator"
version = "1.1.5"
description = "Authenticator in Python"
optional = false
python-versions = ">=3.11"
files = [
    {file = "endpoint.authenticator-1.1.5-py3-none-any.whl", hash = "sha256:cb43f03b69b9f2b8896996e5a0a2e2c6665400f068e27e3516a8fd68fd0f3564"},
    {file = "endpoint.authenticator-1.1.5.tar.gz", hash = "sha256:54cb01834f4490649a901360fe93707438345e4fc37d1bda0f320cf7a7f148f4"},
]
Copy code
[tool.poetry.dependencies]
python = "^3.11"
endpoint-authenticator = {version = ">=1.1.4", source = "endpoint_jfrog"}
h
Hi! I suspect that the issue is related to that 
interpreter_constraints = [">=3.11"]
You always want to upper-bound your interpreter constraints (a future version of Pants may enforce this)
because that open-ended constraint includes 3.12 and maybe 2.13 alphas (I'm not sure what the logic is for those), and 
endpoint-authenticator==1.1.5
may not support those, so the resolve correctly but confusingly fails.
Try 
interpreter_constraints = ["==3.11.*"]
This is not at all obvious, and we probably should never have supported open-ended constraints
so it's an unfortunate foot-gun at the moment
m
Ok giving this a shot! 🤞
@happy-kitchen-89482 unfortunately still getting the same error 😞
Copy code
pants test tests/unit/lambdas/get_configuration/handler_test.py --keep-sandboxes=on_failure -ldebug
Copy code
10:40:45.73 [INFO] Canceled: Building pytest.pex from pantsPytest.lock
10:40:49.30 [INFO] Completed: Building pytest.pex from pantsPytest.lock
10:40:49.84 [INFO] Preserving local process execution dir /private/var/folders/fz/m_mxnw2s4g51nr6402plb7mh0000gn/T/pants-sandbox-cAGFWb for Building requirements.pex with 10 requirements: aws-lambda-powertools==1.31.1; python_version >= "3.11" and python_version < "4.0", boto3==1.26.62; python_version >= "3.11" and python_version < "4.0", endpoint-authenticator==1.1.5; python_version >= "3.11" and python_version < "4.0", nanoid==2.0.0; python_version >= "3.11" and python_version < "4.0", psycopg2-binary==2.9.5; python_version >= "3.11" and python_version < "4.0", pytest-mock==3.10.0; python_version >= "3.11" and python_version < "4.0", pytest==7.2.1; python_version >= "3.11" and python_version < "4.0", requests-mock==1.10.0; python_version >= "3.11" and python_version < "4.0", requests==2.28.2; python_version >= "3.11" and python_version < "4", sqlalchemy==1.4.46; python_version >= "3.11" and python_version < "4.0"
10:40:49.85 [INFO] Completed: Building requirements.pex with 10 requirements: aws-lambda-powertools==1.31.1; python_version >= "3.11" and python_version < "4.0", boto3==1.26.62; python_version >= "3.11" and python_version < "4.0",... (588 characters truncated)
10:40:49.85 [ERROR] 1 Exception encountered:

Engine traceback:
  in `test` goal

ProcessExecutionFailure: Process 'Building requirements.pex with 10 requirements: aws-lambda-powertools==1.31.1; python_version >= "3.11" and python_version < "4.0", boto3==1.26.62; python_version >= "3.11" and python_version < "4.0", endpoint-authenticator==1.1.5; python_version >= "3.11" and python_version < "4.0", nanoid==2.0.0; python_version >= "3.11" and python_version < "4.0", psycopg2-binary==2.9.5; python_version >= "3.11" and python_version < "4.0", pytest-mock==3.10.0; python_version >= "3.11" and python_version < "4.0", pytest==7.2.1; python_version >= "3.11" and python_version < "4.0", requests-mock==1.10.0; python_version >= "3.11" and python_version < "4.0", requests==2.28.2; python_version >= "3.11" and python_version < "4", sqlalchemy==1.4.46; python_version >= "3.11" and python_version < "4.0"' failed with exit code 1.
stdout:

stderr:
pid 15660 -> /Users/rachelellis/.cache/pants/named_caches/pex_root/venvs/839dd85d778df5927a872ace6e92e15b4f301d3d/33f19e3f6ac3abf06260be6310c973dd7cea4c4c/bin/python -sE /Users/rachelellis/.cache/pants/named_caches/pex_root/venvs/839dd85d778df5927a872ace6e92e15b4f301d3d/33f19e3f6ac3abf06260be6310c973dd7cea4c4c/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/rachelellis/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/fz/m_mxnw2s4g51nr6402plb7mh0000gn/T/pants-sandbox-cAGFWb/.tmp/pex-pip-log.1vjw002o/pip.log download --dest /Users/rachelellis/.cache/pants/named_caches/pex_root/downloads/resolver_download.m4wxcf40/Users.rachelellis..pyenv.versions.3.11.8.bin.python3.11 aws-lambda-powertools==1.31.1; python_version >= "3.11" and python_version < "4.0" boto3==1.26.62; python_version >= "3.11" and python_version < "4.0" endpoint-authenticator==1.1.5; python_version >= "3.11" and python_version < "4.0" nanoid==2.0.0; python_version >= "3.11" and python_version < "4.0" psycopg2-binary==2.9.5; python_version >= "3.11" and python_version < "4.0" pytest-mock==3.10.0; python_version >= "3.11" and python_version < "4.0" pytest==7.2.1; python_version >= "3.11" and python_version < "4.0" requests-mock==1.10.0; python_version >= "3.11" and python_version < "4.0" requests==2.28.2; python_version >= "3.11" and python_version < "4" sqlalchemy==1.4.46; python_version >= "3.11" and python_version < "4.0" --index-url <https://pypi.org/simple/> --extra-index-url <http://rellis@endpoint.com>@endpoint.jfrog.io/artifactory/api/pypi/pypi/simple/index --trusted-host rellis@endpoint.com:@endpoint.jfrog.io --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement endpoint-authenticator==1.1.5 (from versions: none)
ERROR: No matching distribution found for endpoint-authenticator==1.1.5
Is someone able to help with this? I've been going through the docs and trying a whole bunch of stuff as I search through this channel but I'm still hunting for a solution. I apologize if this is a repeated issue (and seems that it should be rather straightforward) 🙏
c
Can you get it to work with a more limited interpreter constraints (like 
==3.11.*
? You can also set 
--keep-sandboxes=on_failure
to keep the execution environment around. You can then see exactly what command it's running and try removing pieces from it
actually you might get soemthing useful out of the log. you can try to read the file ("/private/var/..."), or to try the command but with the 
--log ...
argument removed
m
Removing the .add for jfog in pants.toml
Copy code
[python-repos]
indexes = ["http://%(env.JFROG_USERNAME)s:%(env.JFROG_ACCESS_TOKEN)s@endpoint.jfrog.io/artifactory/api/pypi/pypi/simple/index"]
and updating the 
interpreter_contraints
to 
["==3.11.*"]
seems to have worked and I'm no longer seeing the error for 
endpoint-authenticator
, but I'm now getting the same error for a different package, 
aws-lambda-powertools
I thought maybe the installed version of that package doesn't work with the updated python, so I updated the 
aws-lambda-powertools
to the latest version, which didn't solve the issue.
c
Ah hm. Silly question, does it work without Pants? If you create a venv with python 3.11 and try installing all the packages with 
--index-url
set to the jfrog url, does that work?
👀 1
h
@many-iron-92447 did you regenerate your lock file after changing the interpreter constraints?
m
@happy-kitchen-89482 yes
c
I think the answer is going to be something in the pip.log . looking through my command history I've got something similar, but for me it was because a package didn't have a version for python3.6 . It's odd, though, because generating the lockfile implies that Pants was able to find all the packages? do you need to manually enable lockfiles with Pants 2.17?
m
Actually, now that I'm trying to generate the lockfile:
Copy code
pants generate-lockfiles --resolve=pytest
I'm getting this error:
Copy code
ERROR: Could not find a version that satisfies the requirement pytest>=7.0.1 (from versions: none)
ERROR: No matching distribution found for pytest>=7.0.1
c
I'd say that the next step is to either see what's in the pip log or to try to get it to work without pants. Is it possible that the connection to the artifact repo isn't working? maybe firewall or auth?
m
Yeah I'm working through issues in the venv without pants, I think one issue is just that the code is so outdated there are dependency upgrades needed all over the place
Do you know if there's anything special about pants with aws lambdas?
h
One common gotcha in that regard is trying to build the lambda (which runs on linux) on a macos developer machine
6 Views
Open in Slack
PreviousNext
Powered by