Wait, I think the

python_test

should use the

linux_docker

. The docker_environment should be as it is.

Yeah just tested it and that works. So basically the

docker_environment

generates the target using the

name

you provide and other target consumes it using the alias you set for the

name

in pants.toml

vscode ➜ /nefino_li (feature/2087-add-tests-to-services-qgis-export-feature ✗) $ ./pants --no-pantsd --no-local-cache --print-stacktrace -ldebug --version
Bootstrapping Pants using /usr/bin/python3.9
Creating the virtualenv PEX.
Downloading the Pex PEX.
SHA256 fingerprint of <https://github.com/pantsbuild/pex/releases/download/v2.1.103/pex> verified.
Scrubbing PYTHONPATH
Installing pantsbuild.pants==2.15.0a1 into a virtual environment at /home/vscode/.cache/pants/setup/bootstrap-Linux-x86_64/2.15.0a1_py39
Scrubbing PYTHONPATH
New virtual environment successfully created at /home/vscode/.cache/pants/setup/bootstrap-Linux-x86_64/2.15.0a1_py39.
12:28:36.54 [DEBUG] File handle limit is: 1048576
12:28:36.55 [DEBUG] Using [bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }] for process execution.
12:28:37.18 [DEBUG] Launching 1 roots (poll=false).

Not getting any further. @curved-television-6568 Any idea?

We tried on the machine of a co-worker: 1. Pants 2.14 was installed, then successfully upgrade to 2.15.0a1 2. Trying to run a tests hangs on the same line as I had above:

./pants --print-stacktrace -ldebug test services/qgis/tests/example_test.py
10:02:26.00 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f96edb4a9d0>
10:02:26.00 [DEBUG] terminating pantsd
10:02:26.00 [DEBUG] sending signal 15 to pid 3087
10:02:27.01 [DEBUG] successfully terminated pid 3087
10:02:27.01 [DEBUG] purging metadata directory: .pants/.pids/4b81b7112ed6/pantsd
10:02:27.01 [DEBUG] Launching pantsd
10:02:27.01 [DEBUG] purging metadata directory: .pants/.pids/4b81b7112ed6/pantsd
10:02:27.01 [DEBUG] pantsd command is: PANTS_DAEMON_ENTRYPOINT=pants.pantsd.pants_daemon:launch_new_pantsd_instance PYTHONPATH=/home/vscode/.cache/pants/setup/bootstrap-Linux-x86_64/pants.4UZp49/install/bin:/nefino_li/dist/export/python/virtualenvs/default/3.8.15/lib/python3.8/site-packages:/nefino_li:/usr/lib/python39.zip:/usr/lib/python3.9:/usr/lib/python3.9/lib-dynload:/home/vscode/.cache/pants/setup/bootstrap-Linux-x86_64/2.15.0a1_py39/lib/python3.9/site-packages /home/vscode/.cache/pants/setup/bootstrap-Linux-x86_64/2.15.0a1_py39/bin/python /home/vscode/.cache/pants/setup/bootstrap-Linux-x86_64/2.15.0a1_py39/bin/pants --pants-bin-name=./pants --pants-version=2.15.0a1 --print-stacktrace -ldebug test services/qgis/tests/example_test.py
10:02:28.91 [DEBUG] pantsd is running at pid 3786, pailgun port is 45317
10:02:28.91 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f96edb4a9d0>
10:02:28.91 [DEBUG] Connecting to pantsd on port 45317
10:02:28.91 [DEBUG] Connecting to pantsd on port 45317 attempt 1/3
10:02:28.92 [DEBUG] Connected to pantsd
10:02:28.93 [DEBUG] Launching 1 roots (poll=false).

3. Tried again with `--no-pantsd`:

./pants --print-stacktrace -ldebug test --no-pantsd services/qgis/tests/example_test.py
10:04:27.84 [DEBUG] File handle limit is: 1048576
10:04:27.85 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }] for process execution.
10:04:28.37 [DEBUG] Launching 1 roots (poll=false).

4. As still not working we were trying to use the latest successfull commit from 2.15.x branch. But now even

./pants --version

hangs again at this stage All of that is running in a devcontainer with Dockerfile

python-3.8.15-bullseye

also, between attempts, you might consider killing the

pantsd

process: it’s not necessary in general, but something strange is going on here.

if you’re able to share the trace level logs of a

--no-pantsd

run in a DM, that might be helpful.

Okay, with a lot of trial and error I finally found the issue. I have commented out

build_file_prelude_globs = ["pants_plugins/macros.py"]

and then no issue at all. My

macros.py

looks as follows:

def http_service_image(name, service, **kwargs):
    python_awslambda(
        name=f"{service}-lambda",
        runtime="python3.8",
        handler=f"{service}/lambda_handler.py:handler",
        # Otherwise pants cannot safely infer as dir name is the same in all services
        dependencies=[f"//services/{service}/api", f"//services/{service}/lambda_handler.py"]
    )

    docker_image(
        name=name,
        dependencies=[f":{service}-lambda", ":entrypoint"],
        image_tags=["{build_args.IMAGE_TAG}"],
        extra_build_args=[f"SERVICE={service}"],
        repository=f"{{build_args.{service.upper()}_REPO}}",
        **kwargs
    )


def env_vars_django(**kwargs):
    return [
        "ENVIRONMENT",
        "DEBUG",
        "ALLOWED_HOSTS",
        "SECRET_KEY",
        "DJANGO_LOGGING_ENABLED",
        "DJANGO_LOG_LEVEL",
        "DOMAIN",
    ]

When I comment out

http_service_image

, it also works. Any idea what is wrong with that? @curved-television-6568 helped me setting this up here. It perfectly works with

2.14.0

.

@refined-addition-53644 I have updated the environment name in the

python_tests

target as suggested, but now I am getting:

14:37:59.83 [ERROR] 1 Exception encountered:

Engine traceback:
  in `test` goal
  in Run Pytest - (environment:linux_docker, services/qgis/tests/example_test.py)

ProcessExecutionFailure: Process 'Extract environment variables from the Docker image python:3.8-bullseye' failed with exit code 126.
stdout:
OCI runtime exec failed: exec failed: unable to start container process: chdir to cwd ("/pants-sandbox/pants-sandbox-kqW2I5") set in config.json failed: no such file or directory: unknown

stderr:

I can see that the container is being started. It should be noted that I am running pants itself in a Docker container with access to

/var/run/docker.sock

. Do I need to mount something to the pants container as well such that files are correctly mounted from the host to the test container?

https://github.com/pantsbuild/pants/issues/17852 https://github.com/pantsbuild/pants/issues/17851

@witty-crayon-22786 I have added a repo to the issue https://github.com/pantsbuild/pants/issues/17852 which reproduces the error.

@thousands-plumber-33255: hey Chris: thanks again for making the repro repo. it looks like it is private though (or a 404 at least).

@witty-crayon-22786 Have you had a look already?

https://github.com/pantsbuild/pants/pull/17939 : will cherry pick before the end of the day.