Hi folks, quick question. I'm trying to use a

requirements.txt

with a

GIT_TOKEN

env variable that usually get expanded when using pip. For example:

my-lib@$GIT_TOKEN@github.com/my-org/my-lib.git@v1.0.0

but when trying to use the

python_sources

I get a

ValueError

saying it's an invalid URL. What's the best way to tackle this?

Hi! Hmm I can't think of any simple approach because BUILD files do not currently support evaluating environment variables either The best workaround would be to create a light weight "target generator", which does have access to environment variables through the Rules API. Here is an example of a more simple one: https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/plugin_development/pants_requirements.py. You would get access to environment variables like this https://www.pantsbuild.org/docs/rules-api-process#environment-variables You could still keep the same value in your requirements.txt I think, and then you'll in a BUILD file declare something like

workaround_for_git_token()

, i.e. the name of your target generator, which will create a

python_requirement

target I'm happy to give more detailed instructions if that approach sounds viable and you're not sure how to proceed

are there plans for any sort of secret management through pants?

We could specifically support env interpolation in requirements though

Hey just wanted to say thanks for the suggestion @hundreds-father-404. I was able to create a plugin to inject the env variable in the requeriments.txt fairly quickly last week, but it did end up revealing more systemics issues as internal dependencies in other repos started using other conventions (e.g., ssh urls). After switching to ssh I discovered that nested dependencies become a headache as I was getting into lockfile conflicts between versions of different libraries. As a new strategy, I'm testing putting all the deps into a single repo to start resolving 3rparty dependencies more effectively. I switched to pants 2.13.0rc0 to use the PEX support for filepath in lockfiles, but wasn't able to make it work. Is there a specific requirement to integrate the relative path in the lockfile? Currently I'm trying to use

poetry_requirements

with the dependency declared as a path, but I'm guessing for PEX lockfile to work with relative paths I would need to instead declare the dependency explicitly in the

BUILD

file or something similar?

Thanks for the update! For local requirements, we still need to hook Pants up to Pex appropriately via https://github.com/pantsbuild/pants/issues/16416 In the meantime, the work around is to set up

[python-repos].repos

(aka

--find-links

) to host the files, although I'm tbh not totally sure how to do that