https://pantsbuild.org/ logo
#general
Title
# general
r

rhythmic-glass-66959

08/15/2022, 6:39 PM
Hi, I'm trying to copy python sources into a docker image. I added the
python_sources
target to the
docker_image
dependencies, but when building the docker image, I got an error message that the
file does not exist
.
If I use a
files
target, it works...
r

refined-addition-53644

08/15/2022, 6:56 PM
It only supports
file/files
and other package related goals https://www.pantsbuild.org/docs/docker#adding-dependencies-to-your-docker_image-targets
b

bitter-ability-32190

08/15/2022, 6:59 PM
r

rhythmic-glass-66959

08/15/2022, 7:04 PM
@refined-addition-53644 I saw that, but in example-docker, they use a
shell_sources
target. Moreover, the documentation for docker_image target suggests that it is possible...
@bitter-ability-32190, That's great, but is there a workaround in the meantime?
r

refined-addition-53644

08/15/2022, 7:08 PM
I would say then the docs are not well aligned in this case. I feel like docker_image
dependencies
option might just be a copy-paste from another place. @hundreds-father-404 Any thoughts?
b

bitter-ability-32190

08/15/2022, 7:10 PM
Use
--no-process-cleanup
on your
./pants
invocation to see the sandbox the `docker `command is running in
It'll log about a leaked tempdir
(You'll probably want to add a newline or something to the file so you don't hit the cache)
r

rhythmic-glass-66959

08/15/2022, 7:14 PM
There's no python sources in the sandbox 😞
👀 1
The goal of that is to be able to use “editable” installs for libraries.
h

hundreds-father-404

08/15/2022, 7:17 PM
I feel like docker_image dependencies option might just be a copy-paste from another place.
@hundreds-father-404 Any thoughts? indeed. that's pretty easy to fix, and we should be doing it more generally imo. Mind opening an issue?
👍 1
r

rhythmic-glass-66959

08/15/2022, 7:23 PM
Thank you all for your help. But I'm wondering, how do you achieve in pants the installation of your librairies in a docker image without bumping a revision number and building a distribution for each change? This seems to be against the philosophy of a monorepo...
h

happy-kitchen-89482

08/15/2022, 7:43 PM
This is where PEX comes in
The typical idiom is to build a PEX file and copy it into the image
2
@bitter-ability-32190 wrote a post recently with some tips on how to do this performantly: https://blog.pantsbuild.org/optimizing-python-docker-deploys-using-pants/
🙌 1
1
PEX is the “shippable unit” in Pants, because it’s really easy to work with. Pants knows how to build them efficiently, and handles things like following dependencies for you
So you can just say “I want all the transitive first party and third party deps of this entry point” and the right thing happens
r

rhythmic-glass-66959

08/15/2022, 7:53 PM
In my case, I can't use a PEX file, since my image is built from a other image (provided by the vendor). So my Dockerfile looks like that. I was looking for a way to install my Nautobot plugins (which are basically Django applications) in "editable" mode... Make sense?
h

happy-kitchen-89482

08/15/2022, 8:24 PM
Not sure I understand. Why couldn’t you build a PEX containing the results of resolving plugin_requirements.txt ?
r

rhythmic-glass-66959

08/15/2022, 8:42 PM
My understanding is that a PEX file contains the source code and all its dependencies. But in my case, some of the dependencies are already installed in the base docker image... For example, all my plugins depends on Nautobot, but Nautobot is already installed in the provider image.
But maybe I'm missing something here...
h

happy-kitchen-89482

08/15/2022, 9:58 PM
Ah, I see
You can exclude dependencies from the pex IIRC
b

bitter-ability-32190

08/15/2022, 9:59 PM
In the end though, I think Eric's problem is another nail in the coffin of us trying to assume the "types" of things in the dependencies field 😔
h

happy-kitchen-89482

08/15/2022, 10:15 PM
Well that excludes all dependencies, it sounds like @rhythmic-glass-66959 wants to exclude just some.
@rhythmic-glass-66959 are the provided requirements still modeled in your repo (e.g., for tests and things that run outside of that image)?
r

rhythmic-glass-66959

08/16/2022, 12:34 AM
For the plugins we maintain yes, but not the Nautobot core.
I read @bitter-ability-32190 proposal about multiple typed dependencies fields. It's interesting and can surely help to solve my problem...
❤️ 2
b

bitter-ability-32190

08/16/2022, 12:50 AM
The idea is that if we don't try and make assumptions about what the type of a dependency is, and instead let the user guide us, we're much more equipped and ready to handle more complex workflows
👍 1
h

happy-kitchen-89482

08/16/2022, 1:06 AM
Well, in this case, if Nautobot core isn’t a dep in your requirements.txt then the pex won’t include it, surely?
r

rhythmic-glass-66959

08/16/2022, 1:40 AM
Sorry, I didn't quite understand the question. Yes, Nautobot is in my requirements, as of Django, DRF, etc. which are already installed in the base Docker image.
h

happy-kitchen-89482

08/16/2022, 2:08 AM
Ah, gotcha
So Pants installs locally it when you
./pants test
etc. but you don’t want to do that when you deploy?
r

rhythmic-glass-66959

08/16/2022, 3:02 AM
Right
h

happy-kitchen-89482

08/16/2022, 3:15 AM
Hmmmmm, so I wonder what happens if you force-exclude that dep from a
pex_binary
target by prefixing it with
!!
(e.g.,
dependencies=['!!3rdparty#nautobot', '!!3rdparty#django', ...]
etc.
Or whatever the addresses of those requirements are (you can see them with
./pants list ::
)
That should omit them from the pex, I believe
4 Views