runtime -> According to the doc: ” The

runtime

should be one of the values from https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html. ” So basically, it’s the Python version. Also, It is required to define the lambda handler, which, as the runtime, I have to redefine in my IaC tool.

How are you specifying it in your IaC tool? If the violation of DRY was offensive enough, you could write a target generator that reads in the IaC to auto-set the target's fields 🙂

One last thing: Is the venv prompt name configurable?

Regarding the archive thing: part of the reason why type information is necessary is that there is no well defined layout for sources and third-party wheels that could go into the archive. You'd need type information like "this should be a portable virtualenv" (which isn't really a thing), at which point you'd probably not call it

archive

anymore

it is not. https://github.com/pantsbuild/pants/blob/cf172080a5de8140b4e244e15a2f6265498e957e/src/python/pants/backend/python/goals/export.py#L116-L120 what would you want it to be?

In our current setup, we've typically named our venvs as

<repo name>-venv

so that people have awareness if they're using the wrong venv when switching between repos (since we're not fully monorepo integrated yet)

nbd for right now, just wanted to make sure I didn't miss an option in the docs

Main reason this is not configurable is resolves must have their own venvs. We could create a DSL option for people but that's complex

That’s interesting. To answer @hundreds-father-404: I’m using aws cdk. So basically, it’s typescript stuff and for a Lambda, that’s how it works:

const fn = new lambda.Function(this, 'MyFunction', {
  runtime: lambda.Runtime.NODEJS_12_X,
  handler: 'index.handler',
  code: lambda.Code.fromAsset(path.join(__dirname, 'lambda-handler')),
});

It requires a runtime, a handler and an asset. Currently in my case, the asset is the archive file generated by our internal tool. My idea was to replace this asset by one generated by Pants. The idea of a target generator seems interesting. I’ll have to work more on the doc. I just started playing with Pants some hours ago. Do you have any useful resources for this ?

We don't have dedicated examples of target generators yet, which we have discussed. But w/ Pants, core Pants uses the exact same plugin api It would look a bit like the target generator

pex_binaries

, but even simpler because you are generating 1->1 rather than 1->n so no need for things like the

overrides

field https://github.com/pantsbuild/pants/blob/cf172080a5de8140b4e244e15a2f6265498e957e/src/python/pants/backend/python/target_types_rules.py#L77-L124

hey everyone! So I have a quick question. I could be looking at this the wrong way, but we have a multi stage Dockerfile. It follows this layout:

FROM golang:alpine as builder
WORKDIR /app
RUN apk update && apk upgrade && apk add --no-cache ca-certificates
RUN update-ca-certificates
ADD . /app/
RUN CGO_ENABLED=0 GOOS=linux go build -a -ldflags="-s -w" -installsuffix cgo -o my-app .


FROM scratch

COPY --from=builder /app/my-app .
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

ENTRYPOINT ["./my-app"]

However when using the docker integration, it fails saying that

my-app

and

ca-certificates

aren't in the Docker build context. Is there a way to copy files using

--from

with a multi stage build?

Well, that has to happen and is not the reason

I doubt it takes 45 seconds just to materialize, it sounds like it's repeating actual resolve work?

Just a question if anyone has already done this, before I fiddle around with it later today - has anyone tested if the Docker integration works with Podman? Podman claims to be roughly a drop-in, but that hasn't been my experience yet

Does anyone know why the Docker container would be picking up python3.10 instead of 3.9, given that inside the container should only be 3.9. This worked a couple months ago before I installed 3.10 as well on my system. I'm guessing there is a conflict between the Python installed in the container vs the one used to bundle the PEX? That seems like the most obvious first step for me to test against tonight, but if so - is this known? Note: Still on 2.9.x in this project

FROM nginx/unit:1.26.1-python3.9
COPY backend/apigateway/config.json /docker-entrypoint.d/config.json

COPY backend/apigateway/apigateway.pex /apigateway.pex
RUN PEX_TOOLS=1 /apigateway.pex venv /app
WORKDIR /app

16:58:54.10 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building docker image apigateway-container:latest' failed with exit code 1.
stdout:

stderr:
#1 [internal] load build definition from Dockerfile
#1 sha256:dd05a256bf0a700475eee66e2bab7b409dfa252f9d5b9c512a20570645e0a542
#1 transferring dockerfile: 259B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:61b1ab8a8d14acfdedce93b66af15c72891c34a5899e37ffb9c6f9628f63a840
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for <http://docker.io/nginx/unit:1.26.1-python3.9|docker.io/nginx/unit:1.26.1-python3.9>
#3 sha256:29341916069df22232847ba48ae4db61f24890c7b9ecbd26801c5739bc6e7b2b
#3 DONE 0.2s

#4 [1/5] FROM <http://docker.io/nginx/unit:1.26.1-python3.9@sha256:fd65503ce126888587050d2ab77d9324bed7fd5cb84a7da29b926303a3b3beba|docker.io/nginx/unit:1.26.1-python3.9@sha256:fd65503ce126888587050d2ab77d9324bed7fd5cb84a7da29b926303a3b3beba>
#4 sha256:cc3caab62ea82004335d0ff18f5f40e1ae0b0ee6932679be1c7144e8ad4072f7
#4 DONE 0.0s

#5 [internal] load build context
#5 sha256:f07b09714c1c81e44b9534d3b0fdf55d4bbbd51b1d5d730df0f29b1d51ed34a4
#5 transferring context: 17.83MB 4.6s
#5 transferring context: 20.00MB 5.4s done
#5 DONE 5.4s

#6 [2/5] COPY backend/apigateway/config.json /docker-entrypoint.d/config.json
#6 sha256:8663783c824d8e671e1c968f89cc4d12ba24ea91acd6a8508e756eabd479b8bf
#6 CACHED

#7 [3/5] COPY backend/apigateway/apigateway.pex /apigateway.pex
#7 sha256:ba50617df623b8c168123d021d61454e8f26c99e35650d16eb3d614370e5bd53
#7 CACHED

#8 [4/5] RUN PEX_TOOLS=1 /apigateway.pex venv /app
#8 sha256:caaa14b420888270c8c3bff91403db9f760633ee2c61e9340a947bc1761729cb
#8 0.639 /usr/bin/env: 'python3.10': No such file or directory
#8 ERROR: executor failed running [/bin/sh -c PEX_TOOLS=1 /apigateway.pex venv /app]: exit code: 127
------
 > [4/5] RUN PEX_TOOLS=1 /apigateway.pex venv /app:
------
executor failed running [/bin/sh -c PEX_TOOLS=1 /apigateway.pex venv /app]: exit code: 127

I'm looking into macros and

parametrize

to work around my issue with coverage (see here). However I'm not able to use

parametrize

on

extra_env_vars

. Here are the options I tried with their respective errors:

# 1st try:

python_tests(
    name=name,
    extra_env_vars=parametrize(["MODE=1"], ["MODE=2"]),
)

  Exception: Failed to parametrize `//:tests0`:
In parametrize(['MODE=1'], ['MODE=2']:
  Positional arguments must be strings, but `['MODE=1']` was a `list`.

# 2nd try:

python_tests(
    name=name,
    extra_env_vars=parametrize("MODE=1", "MODE=2"),
)

  Exception: Failed to parametrize `//:tests0`:
In parametrize(MODE=1, MODE=2:
  Positional argument `MODE=1` contained separator characters (`=`).

To use `{arg}` as a parameter, you can pass it as a keyword argument to give it an alias. For example: `parametrize(short_memorable_name='{arg}')`

# 3rd try:

python_tests(
    name=name,
    extra_env_vars=[parametrize("MODE=1", "MODE=2")],
)

  InvalidFieldTypeException: The 'extra_env_vars' field in target xyz.py:tests must be an iterable of strings (e.g. a list of strings), but was `[parametrize(MODE=1, MODE=2]` with type `list`.

By the way, a small functionally irrelevant bug in the error output in parametrize.py:59-60, those two lines should be f-strings. (Small PR opened here)

Q27: Now I'm applying pants to my CI. To workaround the error in the first screenshot as instructed, I've changed

./pants --version

to

./pants --version --no-verify-config

, it says another error as in the second screenshot. Why is pants complaining about my pants.toml and how to fix this seem-to-be-bogus error? Here is my pants.toml.

Hey guys 👋 I'm trying to use playwright to run my end to end tests. To install playwright their docs instruct to run

pip install playwright

and then run

playwright install

. I've been trying to use the

experimental_run_shell_command

to run the

playwright install

part but pants runs this before installing the pip dependencies. Can this be done using

experimental_run_shell_command

or should i be using something else?

Hm, thought this was obvious but while I've found a workaround I'd like clarification. We use one package,

jax

, that needs some extras in most contexts (

jax[cpu]

) and in one particular context needs different extras (

jax[mystical_cuda_requirements]

). Honestly this should be true for

torch

as well but the cuda/cpu builds there are hellacious (and this is why we can't have nice things like sub-2GB containers). Anyway. We have pinned versions of packages in a constraints file, but at least a while ago you couldn't put extras in there (haven't tried again under the new resolvers). I generate a

python_requirements

based on that constraints.txt file, but if I have

jax

in the constraints file, it creates a target without any extras which does no one any good. I can't figure out how to have extras in a

python_requirements

rule, so I have a separate requirement as

python_requirement(
    name="jax", requirements=["jax[cpu]==0.3.10"], modules=["jax", "jaxlib"]
)

...and my guess for when I need the cuda version is to create a

name="jaxcuda"

which also provides the same modules and resolve in the necessary target. Maybe. Is there a clean way to handle this sort of situation? I scanned the documentation but got mildly confused... I'd really prefer a "use the CPU unless specifically declared otherwise" rather than two separate "jax" targets which have to be resolved for every client target if we can...

Is there a way to specifically depend on an extras requirement? ie. I have a test that's failing because it needs the pydantic email validator (

pydantic[email]

), which I have in the

requirements.txt

, but dep inference doesn't seem smart enough to pull it in

Q28: I configured the unified requirements to define explicit type-stub packages like this. But running

./pants check ::

still says there is no type stubs found for dateutil, yaml, etc. Am I missing something? Also, should I include type-stub packages in both

requirements.txt

and the

python_requirements()

target?

Q29: I'm going to use

--changed-since

option in GitHub Actions CI. suggestion) The doc says

./pants --changed-since=... lint

but this fails with missing target error and

./pants --changed-since=... lint ::

fails since the

--changed-since

flag and explicit target args are mutually exclusive. I could workaround it by using

./pants lint --changed-since=...

. I believe that the docs should be updated. question) I'm going to use

origin/$GITHUB_BASE_REF

as the argument of

--changed-since

. But it says

fatal: bad revision 'origin/main...HEAD'

in a PR. This may be due to the highly limited fetch-depth (default: 1). How could I fetch the commits that are exactly required for

--changed-since

?

Any thoughts on a Pants-equivalent to Bazel's

exclusive

tag for tests? We have one test which if it doesn't get sufficient CPU resources will become flaky.

I present the new error message if

[python-infer].unowned_dependency_behavior = "error"

, which errors when a dependency cannot be inferred. We're thinking of defaulting to warning or error because this error is hopefully much better than a cryptic runtime

ModuleNotFoundErrror

Feedback welcomed!

what’s the proper way to override options from

pants.toml

file by

pants.ci.toml

file? The use case: I’d like to suppress the warnings for local users, but keep them in the CI builds.

pants.ci.toml

[GLOBAL]
ignore_warnings = []

---

pants.toml

[GLOBAL]
ignore_warnings = ["$regex$."]

It seems to work in CI and locally fine, however, I am curious if I am playing a lottery as I am not sure in what order the options are read and used.

Does pants have a "legal entity" who is the owner/"supplier"? Have to fill out compliance/risk forms 😕

I didn't see any docs regarding dependency cycles. I thought they were kinda kosher, but I'm seeing

CycleException

(only if I use explicit

python_source

though).

a.py depends on b.py depends (for type-checking) on a.py

BUILD.pants

# no error
python_sources()  
# Error
python_source(name="a", source="a.py")
python_source(name="b", source="b.py")
# Also no error
python_sources(name="a", sources=["a.py"])
python_sources(name="b", sources=["b.py"])

This seems... unexpected

How does everyone here handle Docker image promotion in a deployment? Pushing the image is fine, but say you want to use a commit hash/pants.hash. How do you assemble all of the correct hash values?

Following https://www.pantsbuild.org/docs/macros I tried

raise ValueError

in a macro, but am getting

Name 'valueerror' is not defined.

I'm trying to figure out some software update tooling things and looking to get some opinions here. I'm looking to build out a pex binary that is capable of updating itself (kind of like how you could

pip install -U pip

). Has anyone done this before and, if so, how?