Hi Guys, after going through the docs and trying a bunch of things and searching in this channel, I am looking for a solution to my problem below: I have a particular package in the Pipfile as follows

package = {version = "==1.0", editable = true, git = "https://${GITHUB_TOKEN}@github.com/githuborg/package@package-1.0"}

This works fine locally to generate Pipfile.lock, but when trying to package using pants, it keeps throwing this error and I am not able to find a solution:

ERROR: Could not find a version that satisfies the requirement package==1.0
ERROR: No matching distribution found for package==1.0

I am sorry if this is a repeated question and if this is quite straightforward, but can someone help me with this? The env variable

GITHUB_TOKEN

is something that is available in the local environment and will also be available to run in CI as well, as we have a CI pipeline for the project. I tried adding the following in pants.toml to get it to work, but it did not help:

[subprocess-environment]
env_vars.add = ["GITHUB_TOKEN"]

Pants has a great debugging tool for most kinds of process issues like these.

pants --keep-sandboxes=on_failure ...

Run your command with that flag and Pants won't cleanup the sandbox of any Process that failed. You can then try and re-run the command, see the output, twiddle bits, etc... and find out what's truly going on under the hood

Thanks, checking this now

Pants doesn't even try. There is no consultation of either the

"git"

or

"ref"

lock keys here: https://github.com/pantsbuild/pants/blob/b625f09342aadd96a2882a32acce97d285034d64/src/python/pants/backend/python/macros/pipenv_requirements.py#L71-L88 So this is a bug in Pants Pipfile.lock handling @victorious-keyboard-86089.

Oh ok, thanks for letting me know. I was breaking my head wondering what I am doing wrong 😅 But, I also tried by specifically adding a

python_requirement

to

BUILD

as follows:

python_requirement(
    name="package",
    requirements=["package@ git+<https://env>(GITHUB_TOKEN)@github.com/git_org/package@package-1.0"],
    dependencies=[":root#protobuf"],
)

This too kept failing after hitting max retries. I figured that it is not able to get the git url with the correct token for some reason. But I cannot really add the token directly anywhere and have to pass it as an environment variable. How do you suggest I go about this? Unfortunately, we are currently not allowed to use

git+ssh

urls, which we were using previously and was working.

Thrashing to Poetry is unlikely to help here, Pants is still in the middle.

Yes, running this works

pip install package@ git+https://$GITHUB_TOKEN@github.com/git_org/package@package-1.0

I don't believe the

env(...)

function is interpolated in strings by Pants: https://www.pantsbuild.org/docs/targets#environment-variables Have you tried an f-string?

Oops, I might have missed that 😬

Any way you want to construct the string is fine, just don't rely on magic to guess that's a function call buried in there.

Lol genuinely missed it, I tried way too many permutations and I remember it working once, after which it didn’t again. I think now I know why. Will let you know if that works (I think it will now). Thanks!

Ok, great.

So that would mean I run

pants generate-lockfile

and have all dependencies in a requirements file and ditch pipenv?

My comment about

$ENV

support is a bit orthogonal I think. For all I know, pipenv also supports env vars in (portions of) requirements.

Plot twist:

File "BUILD", line 12
    requirements=["package@ git+<https://env>("GITHUB_TOKEN")@github.com/git_org/package@package-1.0"],
                                             ^
SyntaxError: invalid syntax

This is throwing syntax error now

do you just need to escape it?

\"GITHUB_TOKEN\"

Yeah, @victorious-keyboard-86089 you're in knots at this point. That's a bare string again with an embedded env function.

😅

As to the latter, I'm not sure. If the

python_requirements

target works with pip-style

$ENV_VAR

(no use of

env

function), then requirements.txt should also work just fine.

Got it, thanks @enough-analyst-54434