Hi, is it possible to set env vars passed to docker using

build_arg

to have some default value. The use case is we set such env vars only in our CI/CD but for local run it can be any random value. https://www.pantsbuild.org/docs/tagging-docker-images#using-env-vars-to-include-dynamic-data-in-tags

👋 Hi everyone!

Do we expect an option value passed via the command line to be appended to the list of values specified in the

pants.toml

? 😕 I could be misunderstanding the docs.

Is there any kind of default passthrough of environment variables pant backend passes through (e.g.

$HOME

)?

Is there some obvious way to find out which platform my

pex_binary

is built to target? I missed that they are built to target the current host machine and have some issues with docker builds and the fact some of our devs use macos.

I'm finally splitting up a big-ol' BUILD file with lots of recursive targets and overrides. Is there some automated tooling to help with this? Like maybe we could steal some Bazel tools. Or will I need to just need to process this myself?

I'm looking for a suggestion on how to approach a scenario like this: we're building a couple of python apps, packaging them as Docker containers, to be later deployed on a k8s cluster. We're building the apps as pex_binaries, and then simply copying them into the container. I figure, for this to work properly it's best to actually build the apps on the target platform (correct dependencies, even if the prebuilt wheels aren't available). In order to achieve all this, we use a build/publish step in gitlab CI, involving the use of docker-in-docker and a custom builder image based on the same distro as the target images. (We also use dind this way, because our dev environments are all containerized so this limits the use of tools such as docker, buildah or podman to build the images). The steps are then: build all the artifacts (pexes, containers) and publish them. Lately, I encountered a problem when dealing with big 3rdparty dependencies.

ProcessExecutionFailure: Process 'Building...failed with exit code 1.
stdout:
stderr:
pid 536 -> /root/.cache/pants/named_caches/pex_root/venvs/.../pex --retries [...] --retries 5 --timeout 15 exited with -9 and STDERR:
None

Building the same pex_binary works locally, so I assume this is a problem with the gitlab shared runner running OOM (I think it has <4gigs available) So I guess my questions are: 1. Am I correct about it being an oom error? 2. Do you maybe see a simpler alternative to using pants in the scenario I just described?

seeking some basic advice on python project structure and code sharing: currently, we have around 15 deployable units in our mono repo, and the way we prepare the docker containers for deployments is that each project’s dockerfile copies over at least 5-8 of the other projects according to dependencies, then there is a slew of pip installs in the docker container

thats just the pattern that has been used over and over, and we are about to add 3-4 more deployable units

as you may guess, because everything depends on everything else, we have to deploy nearly all these these units together to make sure there aren’t any issues

so the first step we are taking is, try to package internal libraries that get referenced in nearly all the projects, but moving them out of the repo has proved difficult and jarring for the devs

any tips? I am just getting started with experimenting with pants to incrementally layer it in

pants recommended way of packaging 1st party (+ any 3rd party) dependencies is

pex_binary

. Not sure if you already read about it https://www.pantsbuild.org/docs/python-package-goal#creating-a-pex-file-from-a-pex_binary-target You can combine this with docker. Worth going though the docs to understand the flow. https://www.pantsbuild.org/docs/docker

Hi all, I was wondering if there is a general template on how to organize python monorepo with pants? We think of moving our code from multirepo to monorepo, and I was wondering, where to put the requirement files? Our services (and a common library) share the same versions of libraries like pandas etc. One example I saw https://towardsdatascience.com/building-a-monorepo-for-data-science-with-pantsbuild-2f77b9ee14bd?gi=100b8965994b appears to have only single requirements.txt at the root of the monorepo - is that common? I was thinking it makes sense, since pants figures out what packages are required by individual python modules anyway?

Been trying to wrap

pants

invocations in pythons

subprocess

. So far it had worked ok, but using

[python-repos]
indexes = [
    '<https://aws>:%(env.CODE_ARTIFACT_TOKEN)s@<our-python-repo>/simple
]

and adding the appropriate platform to the

pex_binary

causes

subprocess.run(f'./pants package {dockerfile} --dynamic-ui=False', shell=True, text=True, env=os.environ)

to hang. Running

./pants package <the-dockerfile>

works, so it's something specific to

subprocess

Yes I need to study up on this and introduce it, but first I gotta see how you reference code inside of a pex binary, like shared configuration etc etc

What do you mean “reference code” inside pex binary? It just packages all of your python code and its underlying dependencies

The PEX file will contain all the code needed to run the binary, namely:

• All Python code and resources the binary transitively depends on.

• The resolved 3rd-party Python dependencies (sdists and wheels) of all targets the binary transitively depends on.

ah apologies I might be using the wrong terminology since I am newer to python if I copy a PEX binary into a docker container that used to copy the whole project directory, I need to see how that affects the imports in the code base

you don’t need to change any import. It’s like it does all the copying you were doing earlier when building docker. As long as your imports are valid locally it should work inside pex and docker

ah ok! I just need to stand up reference repo and just see how it works, but thanks for giving me some direction

Is there a way to do the equivalent of

GOARCH=amd64 GOOS=linux go build .

when using pant's golang support (`./pants package src/go/someplace/goapp/::`` ) ? didn't find any reference to this in docs https://www.pantsbuild.org/v2.13/docs/go (trying to build a linux amd64 files on an apple m1)

Hey folks, is there a way to have pants use locked dependencies for a

python_distribution

?

Hi, I have a question about imports.

Does anyone else see value in having a target overview page like this? A table of targets per backend? https://docs.bazel.build/versions/main/be/overview.html I personally love this

can I provide the buildkit

--cache-from

argument to the

docker_image

via some pants invocation?

I’ve noticed a small bug in the way Pants takes in configuration options for coverage-py. I’ve created a GitHub issue with a minimal reproducable here: https://github.com/pantsbuild/pants/issues/16760 Thought it best to also notify in this channel. If anybody could take a look at this I’d very much appreciate it. Thanks. 🙂

Hi, this is more of a learner’s question - as in to understand how Pants work: when do the venvs under

~/.cache/pants/named_caches/pex_root/venvs

get created? I see a bunch of them in my linux Docker container where I run Pants, why is that so? Why can’t there be only one venv? Moreover, how are the directories under

venvs/

named? After some SHA256 (of what)? And then sub-directories under them which eventually have pex installed? For example, the last two directories in this path:

~/.cache/pants/named_caches/pex_root/venvs/f61cf2fb294c1349fb246a9500aef689076f209a/319fabea91de21f4b9853622585d1be26b57025e/

has anyone used nix (specifically nix-shell) to provide base layer dependencies for pants execution environment? is that even a good idea?

Hello, I have an issue I can't really pinpoint yet. If I add a new py file in my project and repackage my lambda zip file, the file isn't there. I can see the dependency by doing a

pants dependencies --transitive the lambda_target

but

pants package

doesn't bring it in. If I

rm -r dist/

,

pkill pantsd

and repackage, the file is finally added. Do I miss something? I'm not sure how to diagnose that kind of issue.

Reviving a

tailor

issue I'm running into when creating something with pre-commit: I'd like to just pass filenames from pre-commit, but the issue is stuff in

pants_ignore

. Anything in

pants_ignore

that shows up on the command line causes pants to error out. Example:

❯ ./pants tailor --check aiven/README.md 
14:28:43.11 [ERROR] 1 Exception encountered:

  Exception: Unmatched glob from CLI arguments: "aiven/README.md"

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.13/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.

Is there a way to avoid this? Or do I need to filter before passing to

tailor

?