https://pantsbuild.org/ logo
Join Slack
Powered by
Hi all, I'm migrating a ~1M LOC monorepo from poet...
# welcome
s
Hi all, I'm migrating a ~1M LOC monorepo from poetry to Pants, with the primary goal of faster tests and ancillary benefits of dep mgmt, faster packaging, and introspection
šŸ‘‹ 2
I'm interested in contributing to pants for the following: ā€¢ A Sonar-scanner plugin which simplifies executing code quality analysis ā€¢ A simplified 
docker_image
target which doesn't need any Dockerfile/instructions when packaging a PEX ā—¦ i.e. 
pex_docker_image(source=":pex")
ā—¦ If possible, build the image without the docker daemon (OCI compliant image from chroot, similar to gradle's 
jib
plugin)
šŸ™Œ 1
šŸ‘ 1
f
@steep-eve-20716: Maybe take a look at https://pypi.org/project/pants-backend-oci/? cc @gorgeous-winter-99296
šŸ‘€ 1
I'm not aware of its current state, but it is a OCI backend which presumably does not use Docker.
šŸ‘€ 1
s
Awesome, ty
w
pex_docker_image(source=":pex")
How would that work?
s
It would essentially be the same as the 
docker_image
target, except 
instructions
would default to
Copy code
FROM python:alpine
COPY $PEXFILE ./
ENTRYPOINT python $PEXFILE
w
So, my concern might be a hyper-specified target that fits a single use case, but adding to the public API.
That DOES sound like a great macro though
s
If its an extremely common pattern, then it could be abstracted like so. Google does this with the Jib plugin for gradle, allow devs to build a docker image following standardized patterns (copy Jar and dependencies to a standard dir and set entrypoint)
w
Fair, but there might be more performant examples - as in, pre-extracting to a venv or something. Not trying to be a nay-sayer, as I'm a big fan of contributions to Pants - but (as an example), I wouldn't recommend someone package pex in a docker container like that
I would refer to something closer to this as the Cannonical pex/docker integration (based on how deep down the rabbit-hole you want to go): https://www.pantsbuild.org/blog/2022/08/02/optimizing-python-docker-deploys-using-pants
s
I haven't seen this, its exactly what I was looking for. Essentially I'm just suggesting a target which abstracts the complex best-practices (like 
Multi-stage build leveraging 2 PEXs
) into a single target
w
Yeah, I partially started some parametric tests over here: https://github.com/sureshjoshi/perfanity I use macros to define those "ideal" mechanisms right now (in private repos)
s
Nice!
c
What I would love to see, is documented examples of macros that are used out in the real world, that you can adopt in your projects as needed. To make the packaging of pex files into containers as easy as a single target with a pex dependency, but results in all the nitty gritty details laid out by Josh in that blog post without everyone having to discover it over and over.
Also, I will mention this as it seems you're coming in with a rather large project, that those that have very many targets start to run into staggering memory consumption in the pants daemon, causing it to run slower or at times OOM. The root is thus far not known, but being looked at, so could be resolved at any moment once that has been figured out (don't hold your breath though..), but until then I think it's worth knowing up-front. Most recent thread on this topic: https://pantsbuild.slack.com/archives/C046T6T9U/p1719856232324909
šŸ‘ 1
w
Re: Macros - I was suggesting at some point a page of common macros, or to even one-up that, some in-repo, or out-of-repo (not sure), common macros that a user could pull down based on backend or something. Needing to maintain and namespace macros and targets can get to be a pain, but would be pretty great for new/migrating users
ā¤ļø 1
šŸ‘ 1
Re: OOM - I thought one of the leading possibilities was non-discoverable (or fake) dependencies in the chain somewhere
5 Views
Open in Slack
PreviousNext
Powered by