https://pantsbuild.org/ logo
Join Slack
Powered by
Hi, I'm trying to copy python sources into a docke...
# general
r
Hi, I'm trying to copy python sources into a docker image. I added the 
python_sources
target to the 
docker_image
dependencies, but when building the docker image, I got an error message that the 
file does not exist
.
If I use a 
files
target, it works...
r
It only supports 
file/files
and other package related goals https://www.pantsbuild.org/docs/docker#adding-dependencies-to-your-docker_image-targets
b
r
@refined-addition-53644 I saw that, but in example-docker, they use a 
shell_sources
target. Moreover, the documentation for docker_image target suggests that it is possible...
@bitter-ability-32190, That's great, but is there a workaround in the meantime?
r
I would say then the docs are not well aligned in this case. I feel like docker_image 
dependencies
option might just be a copy-paste from another place. @hundreds-father-404 Any thoughts?
b
Use 
--no-process-cleanup
on your 
./pants
invocation to see the sandbox the `docker `command is running in
It'll log about a leaked tempdir
(You'll probably want to add a newline or something to the file so you don't hit the cache)
r
There's no python sources in the sandbox 😞
👀 1
The goal of that is to be able to use “editable” installs for libraries.
h
I feel like docker_image dependencies option might just be a copy-paste from another place.
@hundreds-father-404 Any thoughts? indeed. that's pretty easy to fix, and we should be doing it more generally imo. Mind opening an issue?
👍 1
b
r
Thank you all for your help. But I'm wondering, how do you achieve in pants the installation of your librairies in a docker image without bumping a revision number and building a distribution for each change? This seems to be against the philosophy of a monorepo...
h
This is where PEX comes in
The typical idiom is to build a PEX file and copy it into the image
✅ 2
@bitter-ability-32190 wrote a post recently with some tips on how to do this performantly: https://blog.pantsbuild.org/optimizing-python-docker-deploys-using-pants/
🙌 1
⏊ 1
PEX is the “shippable unit” in Pants, because it’s really easy to work with. Pants knows how to build them efficiently, and handles things like following dependencies for you
So you can just say “I want all the transitive first party and third party deps of this entry point” and the right thing happens
r
In my case, I can't use a PEX file, since my image is built from a other image (provided by the vendor). So my Dockerfile looks like that. I was looking for a way to install my Nautobot plugins (which are basically Django applications) in "editable" mode... Make sense?
h
Not sure I understand. Why couldn’t you build a PEX containing the results of resolving plugin_requirements.txt ?
r
My understanding is that a PEX file contains the source code and all its dependencies. But in my case, some of the dependencies are already installed in the base docker image... For example, all my plugins depends on Nautobot, but Nautobot is already installed in the provider image.
But maybe I'm missing something here...
h
Ah, I see
You can exclude dependencies from the pex IIRC
h
b
In the end though, I think Eric's problem is another nail in the coffin of us trying to assume the "types" of things in the dependencies field 😔
h
Well that excludes all dependencies, it sounds like @rhythmic-glass-66959 wants to exclude just some.
@rhythmic-glass-66959 are the provided requirements still modeled in your repo (e.g., for tests and things that run outside of that image)?
r
For the plugins we maintain yes, but not the Nautobot core.
I read @bitter-ability-32190 proposal about multiple typed dependencies fields. It's interesting and can surely help to solve my problem...
❤️ 2
b
The idea is that if we don't try and make assumptions about what the type of a dependency is, and instead let the user guide us, we're much more equipped and ready to handle more complex workflows
👍 1
h
Well, in this case, if Nautobot core isn’t a dep in your requirements.txt then the pex won’t include it, surely?
r
Sorry, I didn't quite understand the question. Yes, Nautobot is in my requirements, as of Django, DRF, etc. which are already installed in the base Docker image.
h
Ah, gotcha
So Pants installs locally it when you 
./pants test
etc. but you don’t want to do that when you deploy?
r
Right
h
Hmmmmm, so I wonder what happens if you force-exclude that dep from a 
pex_binary
target by prefixing it with 
!!
(e.g., 
dependencies=['!!3rdparty#nautobot', '!!3rdparty#django', ...]
etc.
Or whatever the addresses of those requirements are (you can see them with 
./pants list ::
)
That should omit them from the pex, I believe
4 Views
Open in Slack
PreviousNext
Powered by