Hi all! :wave: I’m new here and I’m just starting ...
# plugins
b
Hi all! 👋 I’m new here and I’m just starting exploring Pants. 😄 @hundreds-father-404 encouraged me to ask on this channel and hear your thoughts. So, in my team, we use Makefile command (
make deb
) to build a Debian package based on the Dockerfile like it was described in this blog post. I would like to use Pants (probably
./pants package
goal?) instead Makefile command to build a Debian package. My initial thought was to use Pants Docker plugin. There are two plugins: • implemented by @magnificent-hamburger-46133 (works for Pants v1) - stored here, • implemented by @flat-zoo-31952 (working for Pants v2) - discussed here. Is it a good way of thinking or is there a better approach to solve it? Here is the example repository with my problem. Thanks in advance for your opinions! 🙇‍♂️
f
You want to use a docker container to make a deb? Or do you want to build a docker image?
the docker plugin i wrote was mostly for building docker images, not using docker images to build other things
b
Actually both, first step is to build an image and then run a container over that image to build a Debian package.
f
Does the docker image need to vary between builds? it doesn't sound like it to me. I'm not getting that you'd even need
./pants
to do the building part. It might be best to just build an image with the version of python you want and the deb tools you want, and run pants inside that, with its cache directories mounted
but maybe i should ask another question here...what are you using pants for in this scenario? the tool your using to bridge the gap between python and debian packaging is
dh-virtualenv
, which works on virtualenvs. What are you thinking pants will give you here that this setup won't?
b
I wanted to keep my example as simple as I can but maybe it was too simple as it didn’t include many projects. Let me describe what I wanted to do in the bigger picture: • I have multiple microservices stored in separate repositories, they are reusing part of the same code, let’s call it
common
• I wanted to put all microservices into a monorepo and extract the
common
part as a separate project to not duplicate the code across multiple repositories • each microservice has its own Dockerfile, they are almost the same, only copied directories differ, Docker image will need to be build every time the codebase will change • I wanted to use Pants to keep track of dependencies in a monorepo and use it to package each microservice separately into a Debian package using my current solution based on Dockerfile
✔️ 1
f
There's probably a better way to do this but my initial idea would be something like this then: 1. Define
python_distribution
for your projects 2. Write a deb_package target type that depends on that python distribution 3. Write a rule to copy the build wheel from your
python_distribution
into a docker container, and install that wheel into a virtualenv inside the container. Make an entry point script for the container that builds the deb package with dh-virtualenv 4. Write a final rule (that depends on the previous) to run the container with proper mounts that will output a deb
i see problems with this if you intend to have multiple debs generated in this way installed on the same machine, as pants tends to build its artifacts as closures, so your "common" components would be duplicated in different downstream deb pacakges; this may not be a problem for you but i'd watch out for it. Where are these deb pacakges intended to be deployed?
b
Thanks for sharing your idea! I will definitely need to read more of the documentation first to get some knowledge before I will be able to implement it. However, one question came to my mind - do the rules mentioned in steps 3. and 4. still need to use a Docker plugin, or is it completely something different? Sorry for my stupid question, I didn’t reach this point in the documentation yet. 🙈 You are right, the Debian packages are installed on the same machine at the end. As you mentioned, duplication of the
common
within separated Debian packages it’s not a problem for me but I need to be careful with it because the code will be reused across all micro services.
f
honestly i would strongly urge consider alternative deployment models to debian packages if you have infra that you can run yourself. Pexes are great for python, and don't require you to adopt more sophisticated orchestration tools like docker would
as for the docker plugin...you can use my code as a guide (especially for building), but the docker plugin i wrote was about docker containers as the artifact, not as a means to an end really
i'll be hopefully re-writing that plugin soon and properly publishing it, and i'll consider this use case though
👍 1
b
That’s true. Unfortunately, it’s not my decision about the Debian packages. It’s something that I need to stick to as an end product. I will definitely use something different for my pet projects. I’m aware that your plugin has different goal but I will most likely use it as a guidance when I will be writing my own. I will also take a look at the plugin that works for Pants v1. That’s a great news! Let me know once it will be available! Thanks for all your help! 🙇‍♂️
👌🏻 1
f
you can re-use the stuff from the build sections of it; the rules API works on rules of InputType -> OutputType so your
docker run
rule could just rely on the
docker build
rule that builds the image, and that's enough to link the two
by "rely on" i mean "take as input or contain a
Get
request for its output"
h
Hey @busy-lion-5707, checking on how the above is going and if there's anything we can help with?