let me see if I can recreate the cache poisoning outside Pants

Yep!!

That is it

nice deduction Chris

OK.

So yeah, @wonderful-yak-59945, you’ll need to blow away your pants cache for the moment

We’ve got a fix to report upstream, and also we’ll need to change how we model Coursier’s cache

@happy-kitchen-89482 Can you write up a ticket with your instructions; I know what changes will need to be made to the cache behaviour, and will be able to write the fix and tests, but you’ve got better info of how to reproduce.

hi - I'm having issues building a pex binary for different platforms. As a sanity check I'm trying to build a pex binary with

platforms

set to my current platform, which I'm looking up by running

pex --help

on my local system. Building the pex with

platforms

unset succeeds, but building the pex with the platform set to my system's string fails with a missing package error (pasted below). What would be the difference between running

pants package

in these two ways?

ERROR: Could not find a version that satisfies the requirement grpc-google-iam-v1
ERROR: No matching distribution found for grpc-google-iam-v1

Added to the issue: https://github.com/pantsbuild/pants/issues/14577#issuecomment-1050088206

I would think this means nothing is distributed for the platform you have selected and is an issue with what is hosted on the registry this is expected to be pulled from and not specifically Pants.

So this is basically a coursier bug

A platform build assumes the platform is foreign and requires wheels: https://pypi.org/project/grpc-google-iam-v1/#files

This is because Pants (Pex (Pip)) can't cross-build from sources.

Right, but it’s one that we can control for, as long as we key the archive cache based on the repos configuration

There is a solution though - just searching code / docs so I don't lie....

Hello!

A platform build assumes the platform is foreign and requires wheels:

See the warning at https://www.pantsbuild.org/docs/awslambda-python#step-3-run-package for more about this

Ok, the solution is the Pex

--resolve-local-platforms

flag which you can set up either in the

pex_binary.resolve_local_platforms=True

bool field or else globally via

[pex-binary-defaults] resolve_local_platforms = True

As Eric points out though, this will only work for your current experiment on your machine.

Switch to another machine where the platform cannot be resolved to a local interpreter (Linux from Mac or vice-versa), and boom.

To support that case you must pre-build wheels for all targeted platforms and make them available on a custom server you configure pants to look at via

[python-repos]

.

So the good news is that Pants controls the location of the Coursier archive cache (i.e. where all the maven repos get dumped to)

So I suspect if we start making the

COURSIER_ARCHIVE_CACHE

value determined by the values of our repositories config, the fix I proposed yesterday should just work

™

ah ok that makes sense that it would be looking for a built version for the platform. Running with --resolve-local-platforms seems not to be working though likely something I'm doing, but yea makes sense that it wouldn't work as a solution in the long run. and ok, I believe I saw another thread on here about setting that up, or setting up a way to build the package from within Docker for the target system. I'll take a closer look, thank you!

makes sense!

Hi @curved-television-6568 - I'm back with another (hopefully interesting) pants docker issue, was hoping you could help me debug. This is a bit of a weird one. I have a "python base" image which has a pretty standard target:

docker_image(
    name="python_base",
    source="python/Dockerfile",
    registries=["base_images"],
    repository="python",
    image_tags=["internal_build"],
    skip_push=True
)

So this builds and tags

base_images/python_base:internal_build

, and other docker_image targets have it as a pants dependency, and their dockerfiles start with

FROM base_images/python_base:internal_build

. All of this worked, until I switched the python_base's source image, from

FROM ubuntu:20.04

to

FROM amd64/ubuntu:20.04

(I'm building on an M1 mac, and trying to force x86 virtualization in my images). Pants can still build python_base, but it can't build any images that depend on it, instead, docker tries to pull the image from docker.io:

#3 [internal] load metadata for <http://docker.io/base_images/python:internal_build|docker.io/base_images/python:internal_build>
#3 sha256:658dd15cabbf2cf418526dfa6c6ce44508d99eec32bbad5dcc6be686cbc57fe2
#3 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for <http://docker.io/base_images/python:internal_build|docker.io/base_images/python:internal_build>:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

However, I see can the image on my system:

> docker image ls | grep base_images/python
base_images/python                internal_build                       438b7f434621   54 minutes ago   328MB

And I can build the image that depends on base_images/python using a standard

docker build

command, it has no problem finding the image. Any idea where I could look next?

ooh awesome, thanks for the further investigation. sounds like the immediate workaround for us then is: 1. add an explicit jvm_artifact for the problematic jars, and point their

url

at the correct repo manually 2. blow away

~/.cache/pants

(and

~/Library/Caches/Coursier

?) before re-running the resolve is that roughly accurate?

@wonderful-yak-59945 Try #2 first, but if that doesn’t help, try #1

Looks like we’re going to be able to make caching dependent on our

--coursier-repos

configuration, without needing to re-download JDKs every time we change the repos configuration. So Pants will, at the very least, be insulated against the cache poisoning