https://pantsbuild.org/ logo
#general
Title
# general
m

modern-wolf-36228

12/07/2021, 10:54 AM
How do I pass a secret to when building? I have this `Dockerfile`:
Copy code
FROM python:3.8
...
COPY n20-*.whl .

RUN --mount=type=secret,id=pipconfig,dst=/etc/pip.conf \
    python -m pip install --no-cache-dir n20-*.whl
With this `BUILD`:
Copy code
docker_image(
    name="n20",
    dependencies=["src/python/n20:wheel"],
    image_tags=["{build_args.BITBUCKET_BUILD_NUMBER}"],
    extra_build_args=["--secret id=pipconfig,src=pip.conf"],
    repository="recital/n20-dev",
)
The
pip.conf
is actually not there. Do I need to have a
file()
with
pip.conf
? If so, how exactly? Because I added that and it didn't work as well.
c

curved-television-6568

12/07/2021, 12:42 PM
Ahh.. tricky that. The
extra_build_args
are for
--build-arg
options only.. there’s currently no generic
passthrough
args options for docker build.
So this would either require something similar to https://github.com/pantsbuild/pants/pull/13818 or proper support for the various build time args we want to support, such as this secret thing.
m

modern-wolf-36228

12/07/2021, 12:44 PM
so... can't go with this
build outside and copy the site-packages? 🤔
c

curved-television-6568

12/07/2021, 12:46 PM
Hmm.. unfortunate. Will fix asap.
m

modern-wolf-36228

12/07/2021, 12:46 PM
thanks for the link, I'll subscribe
c

curved-television-6568

12/07/2021, 12:46 PM
Thanks for running ice breaker on all missing features, much appreciated (I know how frustrating it can be to hit wall after wall)
❤️ 1
m

modern-wolf-36228

12/07/2021, 12:46 PM
dw, I don't feel frustrated
the feature has 2 months
c

curved-television-6568

12/07/2021, 12:47 PM
awesome 🙂
m

modern-wolf-36228

12/07/2021, 12:47 PM
thanks for all the replies 😗 👍
❤️ 2
c

curved-television-6568

12/07/2021, 2:17 PM
m

modern-wolf-36228

12/07/2021, 2:18 PM
what happens if BUILDKIT is not enabled?
error message informing about it?
c

curved-television-6568

12/07/2021, 2:40 PM
Hopefully..? don’t know. that first line of help text is copy paste from
docker build --help
😛
@modern-wolf-36228 I notice your path to
pip.conf
in the example above uses a relative path. Is that file in your buildroot, so an absolute path wouldn’t be feasible for you, or?
m

modern-wolf-36228

12/07/2021, 2:54 PM
it's on the buildroot yes
c

curved-television-6568

12/07/2021, 2:55 PM
So it would be good if relative paths worked relative to the build file in which it was defined, I guess..
m

modern-wolf-36228

12/07/2021, 2:55 PM
but the example/question was open to accept a solution with non-buildroot
jfyk
c

curved-television-6568

12/07/2021, 2:55 PM
I see.. yeah, I’d like this to work for both cases.
m

modern-wolf-36228

12/07/2021, 2:56 PM
but this pip.conf would make sense on the buildroot
🙌 1
c

curved-television-6568

12/07/2021, 2:56 PM
I’ll fix that.
m

modern-wolf-36228

12/07/2021, 2:56 PM
cool
c

curved-television-6568

12/07/2021, 3:29 PM
I’ve never used the
--secret
feature on docker build.. can you confirm that
--secret=id=name,src=/path/to/file
is OK? (so it doesn’t have to be
--secret id=name,…
(as two args, rather than one) Edit: nvm, it was easy enough to try locally…
I’ll fix that.
Done. 😉
h

happy-kitchen-89482

12/07/2021, 4:02 PM
Is
pip.conf
gitignored?
m

modern-wolf-36228

12/07/2021, 4:03 PM
yes, but I removed the ignore statement to try this
h

happy-kitchen-89482

12/07/2021, 4:23 PM
Makes sense. You can have pants "see" it despite continuing to gitignore it by modifying pants_ignore (https://www.pantsbuild.org/docs/reference-global#section-pants-ignore)
2
m

modern-wolf-36228

12/16/2021, 2:38 PM
how would I modify the
pants_ignore
to just allow a single file that is on the gitignore? 🤔
it doesn't matter, it looks like the
secrets
can look at the ignored files 😗
w

witty-crayon-22786

12/16/2021, 4:15 PM
putting a file in
pants_ignore
with a
!
will re-include it
🙌 1
q

quiet-evening-25363

12/17/2021, 4:01 PM
thank you for adding this feature @curved-television-6568! started using it yesterday and it's very useful. I see you went with making the secrets file path relative to the BUILD file. If I do want to include the secret at the root of my project (or a common location) instead, is there a way to reference the root of the pants project (eg. $PANTS_ROOT/.../secret.json)? My use case is that I have a common cloud API key that I'd like to pass as a secret through multiple Dockerfiles and not have to copy it to the subdirectories. thanks!
h

hundreds-father-404

12/17/2021, 4:03 PM
Hi! It can either be an absolute path or a relative path
But hm @curved-television-6568 that's a good point. Do you think it would be more useful for the path to be relative to the build root, rather than the BUILD file?
q

quiet-evening-25363

12/17/2021, 4:04 PM
hmm maybe I'm confused then, but is there an way to have it be an absolute path, but relative to the pants project root? to avoid having an absolute path from my home directory
c

curved-television-6568

12/17/2021, 4:08 PM
Yeah, good point. I think we could introduce support for ”(buildroot)s/other/part/project/..” for that (with a syntax matching that from pants.toml
Missed a % in there.. (on my phone..)
h

hundreds-father-404

12/17/2021, 4:12 PM
Or should we always make it relative to build root? Keep it simple. I wonder how common it is to have a secret specific to just a single Dockerfile and not shared with others.
c

curved-television-6568

12/17/2021, 4:19 PM
True.. I’m thinking more of least surprise, as I would expect a relative path in a BUILD file to be relative to where it is defined.. the
"%(buildroot)s/…"
clearly tells me when that is not the case…
h

hundreds-father-404

12/17/2021, 4:33 PM
Maybe! That's how the source field works, but dependencies requires the full path
q

quiet-evening-25363

12/17/2021, 4:43 PM
+1, file target dependencies behavior seems reasonable:
For files relative to the current BUILD file, prefix with ./; otherwise, put the full path, e.g. ['./sibling.txt', 'resources/demo.json'].
, assuming the full path is relative to the build root, but any of the suggested options sgtm 👍
👍 1
c

curved-television-6568

12/17/2021, 8:38 PM
q

quiet-evening-25363

12/20/2021, 2:40 PM
awesome, thank you!! will try it out today
👍 1
6 Views