I have tried several way to add image_tags on docker publishing. But it seems hard to set correctly. Am I mssing something?

docker_image(
    name="payhere_dashboard",
    source="Dockerfile.dashboard",
    repository="{build_args.REPOSITORY}",
    registries=["----<http://.dkr.ecr.ap-northeast-2.amazonaws.com|.dkr.ecr.ap-northeast-2.amazonaws.com>"],
    image_tags=["latest", "{build_args.GIT_COMMIT}"],
)

ci.yaml command

run: |
          GIT_COMMIT=$(git rev-parse HEAD) ./pants publish \
          --docker-build-args='[ \
            "REGISTRY=${ECR_REGISTRY}", \
            "GIT_COMMIT=${GIT_COMMIT}", \
            "REPOSITORY=stg/dashboard", \
          ]' payhere/app_services/dashboard:dashboard

pants.toml

[docker]
build_args = ["IMAGE_TAG", "COMMIT_REF", "GIT_COMMIT"]

here is the error trace.

Engine traceback:
  in `publish` goal

DockerImageTagValueError: Invalid value for the `image_tags` field of the `docker_image` target at payhere/app_services/dashboard:dashboard: '{build_args.GIT_COMMIT}'.

The placeholder 'GIT_COMMIT' is unknown. Try with one of: build_args, pants, tags.

Thanks! I'm so sorry they've removed the video. Hopefully it's just an oversight that they'll correct as soon as you've notified them it's missing.

Hey! I've got a bit of an issue with trying to get

console_scripts

to work in a

pex_binary

that I'm not sure how to solve. Essentially I've got a package that has two entry_points; *

server

*

worker

server is the main entrypoint, and spawns workers when requested via GRPC. I can set the server as the main entrypoint in a pex-binary, and that works fine. However; the console_script for the worker is never available. I've also tried this: https://github.com/pantsbuild/pants/discussions/17308, but while it shows the entrypoints they aren't on the path. I would expect this to work if I could install the packages into the pex; but it doesn't quite work either due to how the install happens. Our source layout is

src/
  py/
    a/
      pyproject.toml
      README.md
      a/
       ..
    b/
      pyproject.toml
      README.md
      b/
       ...

With

py/a

,

py/b

being individual source roots (

marker_filenames = ["pyproject.toml"]

). However; when depending on

a:package

and

b:package

the pex cannot be built due to the README.md colliding.

Exception: Can only merge Directories with no duplicates, but found 2 duplicate entries in :

`README.md`: 1.) file digest=d31d7752e98a52694aaa502b3ce9e49ef02aeb2459a4dcad81eba6456567d077 size=855:
...
`README.md`: 2.) file digest=964e4cf6af0fa0dfe069e7cd383792d591eb94ad56d41bd16aad4f72f36d6816 size=3501:
...

Which makes sense. But I don't think there's anything wrong with our setup either. Any thoughts?

This message was deleted.

I’m trying to add a dependency (torch) which is available from PyPI and the custom Pytorch domain. If I add the Pytorch domain to my

pants.toml

with

[python-repos].indexes.add

then run

./pants generate-lockfiles

, the lockfile only has the package resolved from the custom domain, even though there are no wheels available for my platform from that domain, whereas wheels are available from PyPI for my domain. I need the extra index to be able to run my tests on CI. What can I do?

When I run

./pants run ::manage -- runserver

for django. Environment variable from .env or .envrc are not loaded. Is there any way to load env?

🎤 Interested in public speaking about Pants or related topics such as CI, build engineering, testing, code quality, Python, Scala, Go, Rust, etc? Opportunities ahoy on this running GH Discussions thread: https://github.com/pantsbuild/pants/discussions/17322 Feel free to chime in there with more any time!

I have a question about conditionally determining requirements with Pants. We have a

Makefile

that checks

UNAME

and installs either

tensorflow

or

tensorflow-macos

into our

venv

. This isn’t an ideal situation but some of the engineers laptops are M1s while others aren't. I'd like to be able to run something like

./pants test ::

and have the right dependency determined automatically

How are people caching pants in CI (for faster execution)? We’re getting this error now: `MDB_CORRUPTED: Located page was wrong type`:

Exception: Snapshot failed: Failed to digest inputs: Throw { val: Error storing Digest { hash: Fingerprint<c93b26416b75378875d53b1a3a96569ddcbf9ae40b87c5f4cde555f5c625cc71>, size_bytes: 217 }: MDB_CORRUPTED: Located page was wrong type, python_traceback: “Traceback (no traceback):\n <pants native internals>\nException: Error storing Digest { hash: Fingerprint<c93b26416b75378875d53b1a3a96569ddcbf9ae40b87c5f4cde555f5c625cc71>, size_bytes: 217 }: MDB_CORRUPTED: Located page was wrong type”, engine_traceback: [“digest_file”] }

Looking around seems like a few people here this as well in CI. The workarounds seem to be either using

--no-pantsd

(didn’t work) or blowing up the cache dir (not a feasible solution for us).

For context we are running in GitLab CI/CD (but this could be anywhere) in Ubuntu CI executors. We’re storing the cache on each executor. A lot of the times (but not always 🤔) CI fails with that error. What seems to be happening is when one CI job generates the cache but then another job (with a different reqs and lock files) tries to reuse it.

For more context, we don’t see this error when using S3 as the cache mechanism. But our cache is in GBs and up-/down-loading the cache takes too long.

After hitting my head my against the wall (ouch) for a few days, I’m looking into REAPI now: https://www.pantsbuild.org/docs/remote-caching-execution Is this the recommended way forward? I’m highly interested in REAPI’s ability to just download what a CI needs.

It probably is, yes. The Pants repo itself uses this in its CI. In that case the backend is provided by Toolchain.

https://www.toolchain.com/ for a Pants-friendly implementation

Thanks. Currently leaning towards “build” on the build-vs-buy spectrum. Any self-hosted suggestions? Wondering if bazel the way to go here.

The page you found above has a few options listed like BuildBarn

I went ahead and tried the docker option in https://github.com/buchgr/bazel-remote it works extremely well, reduced our builds by an order of magnitude.

Happy to jump on a call to share the details.

And I also have a follow up Q: it seems like an REAPI server can only be used to cache

PANTS_NAMED_CACHES_DIR

. Is this right? using S3 to cache

PANTS_SETUP_CACHE

can’t be cached since pants needs to actually install itself and

PANTS_LOCAL_STORE_DIR

are test results which don’t need to be cached?

You're right that pants setup can't be cached, for bootstrapping reasons, but what is cached over REAPI is the contents of the local_store_dir.

The named caches are specifically a local optimization

Possible to specify platforms for all

pex_binary

targets in the project in

pants.toml

?

Is it possible to use the

--debug-adapter

when running the target in a

docker_environment

?

Is there a good alternative to targeting

pip.conf

for Pants rather than using

.pants.rc

? I understand that

pip.conf

isn’t supported because of PEX but is there a clean workaround?

I am a bit confused about the caching mechanism behind pants. I have a file with some unit tests, and if I run the tests with

pants test --debug path/to/test_file.py

The tests run just fine. If I however run them with

pants test ::

I get failing tests, which I have already removed from the test file. What am I missing here?

I have a use case where I’m struggling to understand the proper process using pants. I have packages A and B. I want to publish B as a whl - I’m using

python_distribution()

for that - and I have package A which depends on package B locally. I’m struggling to figure out how to get A to depend on the output of B - I could add B’s source files to A’s resolve but then A wouldn’t inherit B’s 3rd party dependencies. Depending on the generated whl doesn’t seem to be working (not even sure if it should). Package A is using poetry and pants is using

poetry_requirements()

. Happy to provide further info but I’m not sure what the “right” way to do it is to know which errors/logs to share

never had to do it before, but can one specify in the dependencies of a target that it depends on all targets in the subtree, recursively? 🧵

Error running

./pants

on Ubuntu running in VirtualBox I'm trying to troubleshoot some issues a colleague is having and he uses Linux. I created a Ubuntu VM and tried to run Pants but encounter an error on bootstrap. Any obvious advice to resolve this?

vagrant@linux:~/dev/optios/neuroedge$ ./pants version
Bootstrapping Pants using /usr/bin/python3.8
Creating the virtualenv PEX.
Downloading the Pex PEX.
./pants: line 277: curl: command not found
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '/home/vagrant/.cache/pants/setup/bootstrap-Linux-x86_64/pants.KbyQlX/pex'
SHA256 of <https://github.com/pantsbuild/pex/releases/download/v2.1.103/pex> is not as expected. Aborting.

Hi all, I’m trying to sort out some dependency management between first party code. I have a sample repo here: https://github.com/bryanwweber/pants-multi-deps The short version is I have a module

ops_core.data_io.pathlib

from which a function is imported, and a client

client_code.client

that uses the import. However,

pants

warns me that the dependency can’t be inferred. Any help is appreciated!

The environment field has been added to the

pex_binary

target recently. When I specify a local environment where envs are set thourgh

subprocess_environment_env_vars

I am still seeing the host envs and those of the environment. Is this intended?