https://pantsbuild.org/ logo
#general
Title
# general
w

wooden-dentist-97629

08/08/2022, 1:16 AM
Hi folks, quick question. I'm trying to use a
requirements.txt
with a
GIT_TOKEN
env variable that usually get expanded when using pip. For example:
Copy code
my-lib@$GIT_TOKEN@github.com/my-org/my-lib.git@v1.0.0
but when trying to use the
python_sources
I get a
ValueError
saying it's an invalid URL. What's the best way to tackle this?
h

hundreds-father-404

08/08/2022, 3:05 AM
Hi! Hmm I can't think of any simple approach because BUILD files do not currently support evaluating environment variables either The best workaround would be to create a light weight "target generator", which does have access to environment variables through the Rules API. Here is an example of a more simple one: https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/plugin_development/pants_requirements.py. You would get access to environment variables like this https://www.pantsbuild.org/docs/rules-api-process#environment-variables You could still keep the same value in your requirements.txt I think, and then you'll in a BUILD file declare something like
workaround_for_git_token()
, i.e. the name of your target generator, which will create a
python_requirement
target I'm happy to give more detailed instructions if that approach sounds viable and you're not sure how to proceed
f

freezing-lamp-12123

08/08/2022, 6:14 AM
are there plans for any sort of secret management through pants?
h

happy-kitchen-89482

08/08/2022, 12:15 PM
We could specifically support env interpolation in requirements though
👍 1
w

wooden-dentist-97629

08/12/2022, 11:08 PM
Hey just wanted to say thanks for the suggestion @hundreds-father-404. I was able to create a plugin to inject the env variable in the requeriments.txt fairly quickly last week, but it did end up revealing more systemics issues as internal dependencies in other repos started using other conventions (e.g., ssh urls). After switching to ssh I discovered that nested dependencies become a headache as I was getting into lockfile conflicts between versions of different libraries. As a new strategy, I'm testing putting all the deps into a single repo to start resolving 3rparty dependencies more effectively. I switched to pants 2.13.0rc0 to use the PEX support for filepath in lockfiles, but wasn't able to make it work. Is there a specific requirement to integrate the relative path in the lockfile? Currently I'm trying to use
poetry_requirements
with the dependency declared as a path, but I'm guessing for PEX lockfile to work with relative paths I would need to instead declare the dependency explicitly in the
BUILD
file or something similar?
❤️ 1
h

hundreds-father-404

08/13/2022, 1:05 PM
Thanks for the update! For local requirements, we still need to hook Pants up to Pex appropriately via https://github.com/pantsbuild/pants/issues/16416 In the meantime, the work around is to set up
[python-repos].repos
(aka
--find-links
) to host the files, although I'm tbh not totally sure how to do that
4 Views