The tiny plugin approach seems cleaner and is working at this point so I'm gonna head down that road, but thanks for showing me this neat workaround.

New question: With this tiny plugin approach and in macros, how do I get access to config file options?

Am I right that we don't today have an easy way to check for "does a lockfile need regenerating because of changes to its input requirements"? Seems like that could be useful

Actually regenerating to see if there are any differences is not a good solution because changes to the world will cause differences even if the requirement inputs haven't changed.

an attempt to consume the lockfile will fail and indicate that you need to regenerate it. but no, there isn’t an explicit operation currently.

unfortunately, consuming options currently requires a

@rule

. we definitely need to clean up the boundary between

macros

and

@rules

before stabilizing the plugin API.

are you trying to consume global options, or new options that you have declared?

https://github.com/pantsbuild/pants/issues/15723 has some previous discussion

Ah, so it sounds like we're almost all the way there (since we are able to fail that attempt to consume, we must already have the ability to tell)

Thanks for the link @sparse-lifeguard-95737! That is exactly what I'm after

afaik, the hardest part is what the UX should look like. As is common w/ Pants tickets 🙂

Global options, but that answers my question. Looks like you need to "request" the options before you can use then, which you can only do from a

rule

?

that’s right. which is because when options values change,

@rules

are invalidated. and the synchronous API of macros doesn’t have a way to declare its dependencies (…and thus isn’t allowed to have dependencies currently)

If somebody spends some time thinking about UX changes to lockfile generation, https://github.com/pantsbuild/pants/issues/15704 is another one to keep in mind 😄

I have heard 0 complaints about the cargo UX for this. I'd steal.

Gotcha, that makes sense. I have a quick hack in place with a value hardcoded just to demonstrate my point for a POC I'm working on, so I'm good for now. If we make it through the POC with pants I'll revisit this in the future. Thanks for your help with this!

My last question for the day: Do

docker_image

targets only support dependency inference on paths that "look like" the outputs of

pex_binary

targets? Or, can you directly put target addresses anywhere in the Dockerfile and have the dependency inference work? So if, for example I had a Dockerfile with:

FROM path/to/my/parent/docker/image:target

RUN pip install path/to/a/python_distribution:target

Would pants infer the dependency on both the parent docker image and the python package automatically, and inject the output paths for those targets into the build context automatically? I'm digging through the source code now but I honestly can't really figure out if it supports anything beyond

FROM target_address

and

COPY output_path_that_can_be_mapped_to_a_target

sure thing. thanks for the patience. i’ll make sure we have a bug filed for the options-access aspect of this.

has anyone ever run into this error?

ImportError: No module named __pants_df_parser

it’s started popping up ~relatively often in our CI today. it goes away if we retry the job so I don’t think our GHA cache of the pants bootstrap/setup dir is corrupted

Good morning Pants community. I have a question on logging. Is there a way to direct pants outputs to log files instead of or in addition to sending to stdout and stderr? For example, I'd like to log stats but have it elusively go to a logfile.

If we have a published distribution proj/lib and an app proj/app that depends on lib, but wants to lock down an older version (not keep up with the monorepo), what is the best pattern to use for this so Pants always runs, tests and packages the app with the published version of the distribution and not the first party local?

I'm following "pex in a container" here. But I'm seeing the stage for the 3rdparty unpacking not re-used when only changing 1stparty like suggested. I'm assuming this is because the PEX for the 3rdparty stage isn't re-used because it also contains the 1srtparty code. Am I understanding this correctly? Is there a way to have re-use (like mapping the docker build context's PEX, not copying)?

I’ve been facing this odd error when trying to run

./pants lint terraform::

16:47:26.09 [ERROR] 1 Exception encountered:

  Exception: Can only merge Directories with no duplicates, but found 2 duplicate entries in :

I have multiple modules in different folder that have different names but the files within share the same names such as “main.tf”, “variables.tf” and “outputs.tf”. No idea what to do.

Can a plugin override a pants rule? We would like to override

DeployJar

rules instead of adding a new goal or new target. Is this possible?

Is it possible to get a pip friendly version of Pant's lockfiles? When the lockfile is regenerated after a new input or package upgrade, it would be easier to do something along the lines of pants export_constraints && pip upgrade rather than pants export venv && copy/replace.

Is there a way to export the tools together into the

python-default

environment? For instance, when using Vim, I need to activate the venv for the Vim ALE/LSP plugins to auto-detect and utilize the tools like flake8, black, etc.

When using a separate

[python-repo].repos

for a custom find-links option to fetch custom prebuilt

grpcio

wheels (built with linux-aarch64 and macos-arm64) and I run

./pants generate-lockfiles

in macOS, it tries to fetch the linux-aarch64 whl file and says:

OSError: pid 67995 -> /Users/joongi/.cache/pants/named_caches/pex_root/venvs/3865e68cbdb1d654d6cca639c59b5dfbc8be0dda/f57e61ad0b603cc5d644c45312fc4081d6790c14/bin/python -sE /Users/joongi/.cache/pants/named_caches/pex_root/venvs/3865e68cbdb1d654d6cca639c59b5dfbc8be0dda/f57e61ad0b603cc5d644c45312fc4081d6790c14/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/joongi/.cache/pants/named_caches/pex_root/pip_cache --log /Users/joongi/workspace/bai-grpcio/.tmp/process-executionSNsh5K/.tmp/pex-pip-log.h1stom78/pip.log download --dest /Users/joongi/.cache/pants/named_caches/pex_root/downloads/fingerprint_artifact.8wtp11jr --no-deps <http://localhost:8000/grpcio-1.47.0-cp310-cp310-manylinux_2_17_aarch64.whl> --index-url <https://pypi.org/simple/> --find-links <http://localhost:8000> --trusted-host localhost:8000 --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: grpcio-1.47.0-cp310-cp310-manylinux_2_17_aarch64.whl is not a supported wheel on this platform.

The find-links target is served from

python -m http.server

from a directory that contains:

-rw-r--r--@ 1 joongi  staff   2981258 Jul 13 23:10 grpcio-1.47.0-cp310-cp310-macosx_12_0_arm64.whl
-rw-r--r--  1 joongi  staff  62209358 Jul 14 01:39 grpcio-1.47.0-cp310-cp310-manylinux_2_17_aarch64.whl
-rw-r--r--@ 1 joongi  staff   1941172 Jul 13 23:10 grpcio_tools-1.47.0-cp310-cp310-macosx_12_0_arm64.whl
-rw-r--r--  1 joongi  staff  31484142 Jul 14 01:39 grpcio_tools-1.47.0-cp310-cp310-manylinux_2_17_aarch64.whl

How could I let it resolve the correct wheel?

There are quite a few data science companies using Pants. Are any of you using it for parts of the MLops workflow other than strictly build/CI? It'd be really interesting to hear about how you're applying Pants' considerable adaptability in order to handle tasks beyond what we envisioned, and for us to learn how we could serve you better at that!

I got a new error from building a pex.

Failed to digest inputs: "Error storing Digest { hash: Fingerprint<1872bfe7f1b4c4887a4c1381800062a96d980dea39f700b8ac5cebd0ea613ffe>, size_bytes: 104002766 }: MDB_MAP_FULL: Environment mapsize limit reached

Only thing I could find was a really old bug ticket. Is the proper way to resolve this to delete the cache? This was in our CI system so I suspect our cache just finally got big enough to cause issues (though I'm surprised it happened on the worker it did since we haven't had to clear caches for other workers that are running lots of things like unit tests).

Hi! In which cases do we need to add

@resolve=...

?