What does --filter-granularity=file do, exactly? I noticed that when I pass this in, I still get targets in the form of

path/to/file.py:target_name

. What I’m looking for is a way to filter out all of the targets returned such that I only get things like

path/to/file.py

.

thanks, that makes sense! and in that docker image you included your first-party code as well as the requirements file? how did you generate the requirements? was it simply the same set of reqs you have in your entire project?

path/to/file.py:target_name

is the "proper name" of a file-level target (generated by

path/to:target_name

)

Are you looking for file names to pipe into a non-pants cli app?

Yup, exactly. I want to manually pass to pytest because currently pytest-xdist isn't supported via pants

Yeah we had a common requirements.txt for whole repo

Gotcha. So you can do something like:

./pants  --filter-target-type=python_test peek ::

and pluck out the sources with

jq

or some json-parsing script

Note that in Pants 2.13 you can pass the

--filter

arguments to any goal

peek

in this case

if you're on an earlier version you can pipe

./pants filter

to

./pants peek

But this is a bit elaborate

Is Pants's build-in test parallelization not sufficient for your case?

Hey folks - I'm trying to shake out the interaction between

[docker].env_vars / tools

, and a docker context which uses a docker host via ssh. I'm able to see that the local process is spawned for docker, and PATH is set to a private .shims/bin.

ssh

is defined within

[docker].tools

, but I am getting

docker: error during connect: Post "<http://docker.example.com/v1.24/containers/create>": exec: "ssh": executable file not found in $PATH.

The basic test (symlinking, and

env -i

) works trivially.

Is Pants’s build-in test parallelization not sufficient for your case?

It’s not, at the moment. I initially was looking to try to implement https://github.com/pantsbuild/pants/issues/15026, but encountered some issues with pytest-xdist, and wanted to try implementing a workaround in case it ended up being difficult to implement the pytest-xdist feature into Pants. (Issue I ran into is described in the thread on https://pantsbuild.slack.com/archives/C046T6T9U/p1658777475033269.)

I’m not sure if I’m encountering a bug with

--filter-target-type=python_test

. First, I tried running the following.

./pants --changed-since=origin/master dependees --transitive --closed

This output a huge list of changed targets and corresponding dependees (with many test and non-test targets). Then, I ran the following.

./pants --changed-since=origin/master dependees --transitive --closed --filter-target-type=python_test

This output a pruned list of the above, but seems to have overpruned. There existed tests in the first list, that did not exist in the second list. Do you know what might be happening?

any consideration to moving away from slack? maybe use matrix (or setup your own homeserver). notice yall are on the free plan

Yes, we're definitely thinking about this, for several reasons. Zulip has been discussed I believe. @busy-vase-39202 has more details. What's your thinking on this?

(@hallowed-plastic-91664 that last question was for you... :) )

I'm trying to build a docker image for which I expect to add dependencies to the context. It works as intended with a pex binary, but can't figure out how to do it with a python_distribution target. In my

docker_image

definition, I'm adding the python_distribution

src/python/dep1:dist

to the

dependencies

. But in my Dockerfile:

COPY src.python.dep1/dist.whl /wheels/dep1.whl

returns a

COPY failed: file not found in build context or excluded by .dockerignore: stat src.python.dep1/dist.whl: file does not exist

Not clear in here how the syntax should be, is

.ext

supposed to be

.whl

in this context ?

How do you execute unit-tests that require third-party applications? We have a monorepo with go and python microservices and some tests of some services do need a PostgreSQL instance and/or other service(s) to be run before the execution. With classic CI/CD approach we just define these dependencies (db, other services) in

docker-compose

file, where required, and run

docker-compose up

prior to executing

pytest

, for example. So the layout of monorepo looks roughly like this:

└── services
  ├── go-service-1
  │  └── <CODEBASE>
  │  └── docker-compose.yaml
  ├── py-service-1
  │  └── <CODEBASE>
  │  └── docker-compose.yaml
  └── py-service-2
  │  └── <CODEBASE>
    └── docker-compose.yaml

But I don’t see a clear way for

pants

to understand that it need to execute

docker-compose up

in a directory

path/to/service/foo

upon a code-change of that service foo And the use-case is quite common, so perhaps it’s solved somehow differently? Maybe I need to employ

experimental_shell_command

to spin up DBs and set some envs for unit tests to consume? Basically, that’s the only stopping factor of us starting to adopt

pants

— i simply don’t understand how to solve my issue at hand..

Hello! First of all - thank you for building Pants - it is a trully awesome tool. I'm evaluating the use case where all 3rd party packages need to be installed from the internal devpi index. This all works when I manually create a lock file, point a tool (like black) and run

./pants generate-lockfiles ...

However when I try to use a tool that relies on a default lockfile, I get this error: (in this particular case I was running

./pants package

against a

python_distribution

target)

13:07:24.22 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building build_backend.pex from setuptools_default.lock' failed with exit code 1.
stdout:
...
 pex.result.ResultError: There were 2 errors downloading required artifacts:
1. setuptools 57.5 from <https://files.pythonhosted.org/packages/4b/b9/71687c5d2034c863db1db2e0704f5e27581ff3cb44d7f293968c5e08ceb3/setuptools-57.5.0-py3-none-any>.
whl
    <urlopen error [Errno 101] Network is unreachable>
2. wheel 0.37.1 from <https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl> 
    <urlopen error [Errno 101] Network is unreachable>

As it tries to download from https://files.pythonhosted.org it's clear that the resolve doesn't honor the internal package index. I've tried adding the following to my

pants.toml

config with no luck:

[python-repos]
indexes = [
  "<http://devpi.company.internal/root/+simple/>",
]
[subprocess-environment]
env_vars.add = [
  "PIP_INDEX_URL=devpi.company.internal/root/+simple/,
  "PIP_TRUSTED_HOST=devpi.company.internal",
]

Could you maybe point me in some direction where the problem may happen? Thanks very much in advance!

I haven't used Zulip but based upon the public documents and selling points. Seems like it's great for organizing within a corporation. But for an open source project, I feel the "pain points" they talk about are non-existent. Although I do agree that much of the conversation/knowledge is lost in these types of chat applications. In the past, if I had a problem with an open source lib. I could easy just search for it and it might link to an open issue or a forum post. Now for each problem, I might be lucky to see a similar issue but for the most part is now resolved by going directly to the project's preferred chat platform and asking the question there.

I really like matrix. You can bridge other platforms like slack into it to ease the transition. They've also finally added threading support which is in beta.

@proud-dentist-22844 that is why i love using matrix as well. I have my own homeserver and have bridged all of my social media accounts into my homeserver. No more having IG, LinkedIn, FB Msg, WhatsApp, Signal, Telegram installed on my phone

the only exceptions i have had so far is with slack (due to some servers using the free edition due to 10 app limit) and discord

Eventually, I'll setup a home server.

But it works well via element as well

the only downside might be some loss of native features (stickers?) but I can personally live without it

For pants, they would start by creating a matrix space and then a series of matrix rooms in that space analogous to the channels here in slack.

Shot in the dark but maybe you can save me some work: Running pants in aws codebuild, I use custom caching of the folder

cdk.out

, which apparently means symlink shenanigans:

14:47:31.25 [INFO] Initializing scheduler...
14:47:31.43 [INFO] Scheduler initialized.
14:47:31.97 [ERROR] 1 Exception encountered:
Exception: Failed to read link "/codebuild/output/src157654992/src/cdk.out": Absolute symlink: "/codebuild/local-cache/custom/34f023ad3ffa4b6cff760e0246a80119a65a53c5011ff63d5e880846cbd15928/cdk.out"