It appears that `./pants dependencies --filter-ta...
# general
r
It appears that
./pants dependencies --filter-target-type=<target_type> <some_target>
applies the filter to
some_target
, not the list of dependencies, which I find very unintuitive. For example, in directory
db/sample_project_1
I have a
BUILD
that looks like this:
Copy code
python_sources(
    name="lib",
    sources=["*.py"],
    dependencies=["3rdparty:dbr-9.1-reqs", "db/shared:lib"],
    resolve="databricks",
    tags=["model"]
)
When I call
./pants dependencies db/sample_project_1/__init__.py:lib
I get the following:
Copy code
3rdparty/requirements-dbr-9.1.txt:dbr-9.1-reqs
3rdparty:dbr-9.1-reqs#boto3
3rdparty:dbr-9.1-reqs#pandas
3rdparty:dbr-9.1-reqs#scikit-learn
3rdparty:dbr-9.1-reqs#scipy
db/shared/__init__.py:lib
db/shared/config.py:lib
db/shared/runner.py:lib
db/shared:reqs#request
db/notebook_utils/__init__.py:lib
which is what I expect. If I do
./pants dependencies db/sample_project_1/__init__.py:lib --filter-target-type="python_source"
, then I get the exact same output. If I do
./pants dependencies db/sample_project_1/__init__.py:lib --filter-target-type="python_requirement"
, then I get nothing. It appears that this is because
db/sample_project_1/__init__.py:lib
itself is a
python_source
I can see how this behavior might be expected if the goal is to filter targets from a glob (e.g. only get dependencies for
python_source
in
::
), but it seems rather unintuitive to me. The question I want to answer here is: “which of this files dependencies are 3rd party requirements vs other sources” and it looks like the best way to do that is something like:
./pants list --filter-target-type=python_requirement $(./pants dependencies db/sample_project_1/__init__.py:lib | tr '\n' ' ')
On a related note, it would be really nice if it was possible to use
--output-file
in append-mode.
h
To get the side note out of the way, can you open a ticket for an append mode for
--output-file
?
👍 1
As for your main thing - yeah,
--filter
is a general-purpose option for filtering the inputs of a goal, not the outputs (after all, not all goals have outputs that are themselves targets, that is specific to certain introspection goals)
One way to do what you want is to pipe to another pants process:
(waits for Pants native code to compile to test out example)
Copy code
./pants dependencies db/sample_project_1/__init__.py:lib | xargs ./pants --filter-target-type=python_requirement list
r
I tried that as well, and it’s super slow since it has to spawn many pants processes. This one from my post is much faster since it only spawns one additional process:
./pants list --filter-target-type=python_requirement $(./pants dependencies db/sample_project_1/__init__.py:lib | tr '\n' ' ')
Is there a reason xargs is preferred?
h
It should only need to spawn one pants process
Well, two (the original, and one extra in xargs)
my guess is your shell aliases set
xargs = xargs -n 1
or something like that?
Try
xargs -n 5000
r
Ah yes, I had
xargs -I {} ./pants -lerror list --filter-target-type=python_requirement {}
which would definitely do it