Hello wave I have question regarding publishing python packa Pants #general

Hello :wave: I have question regarding publishing...

rich-london-74860

12/05/2023, 5:36 PM

Hello 👋 I have question regarding publishing python packages to a private pypi repository. In CI/CD, the URL for this pypi repository and the basic auth credentials to access it are stored in environment variables. Conventionally, without

pants

, we publish packges using

twine

(which I believe

pants publish

uses as well), which uses the environment variables

TWINE_REPOSITORY_URL

TWINE_USERNAME

TWINE_PASSWORD

. When using

pants

, it’s not possible to use environment variables in the repositories parameter of python_distribution, so instead we dynamically generate a

~/.pypirc

file with values pulled from environment variables. In the

.pypirc

file, we defined an index alias

Copy code

[distutils]
index-servers = artifacts

[artifactory]
username = xxxx
password = xxxx
repository = <https://artifacts.our-host.com/some/path/pypi/pypi-local>

Therefore, in the

python_distribution

target,

repositories=["@artifacts"]

. With this setup

pants publish

publishes packages to our private pypi repository as expected. Recently a bug in CI stopped generating this

.pypirc

file. Now there is no

.pypirc

file, but the

TWINE_*

environment variables are still defined, but somehow

pants publish

still works? If there is no

.pypirc

file, then how does

pants

know what

@artifacts

is? Is

twine

defaulting to the environment variables? If that is the case, then could any dummy alias be set for

repositories

- e.g.

repositories=["@null"]

(

pants publish

will skip a package entirely if

repositories

is not set at all)

curved-television-6568

12/05/2023, 5:46 PM

this is relevant: https://www.pantsbuild.org/docs/python-publish-goal#environment-variables

curved-television-6568

12/05/2023, 5:47 PM

in short, if you have

TWINE_PASSWORD

etc, that will be used, but you can have per repo settings using

TWINE_PASSWORD_ARTIFACTS

etc..

curved-television-6568

12/05/2023, 5:49 PM

but since you say it worked, was the URL in the env as well..?

rich-london-74860

12/05/2023, 5:50 PM

The environment variables

TWINE_REPOSITORY_URL

TWINE_USERNAME

TWINE_PASSWORD

are defined

curved-television-6568

12/05/2023, 5:51 PM

yea, so that would be picked up as “defaults” for any repo alias

curved-television-6568

12/05/2023, 5:51 PM

unless there was also repo specific env vars provided..

curved-television-6568

12/05/2023, 5:52 PM

which is a pants specific way.. that is, pants looks for env vars based on the repo alias, and hands them to twine as

TWINE_USERNAME

etc…

rich-london-74860

12/05/2023, 5:52 PM

Was this added recently? I swear this did not work before

curved-television-6568

12/05/2023, 5:52 PM

not recently, no

curved-television-6568

12/05/2023, 5:53 PM

there was an issue with the repo name override thing, but the rest has been there since implemented

curved-television-6568

12/05/2023, 5:53 PM

and that was fixed over a year ago..

rich-london-74860

12/05/2023, 5:54 PM

Perhaps what threw me off here before was using an undefined alias

🤷 1

3 Views

Open in Slack

Previous Next