@agreeable-microphone-98219 which version of Pants, and in the resulting lockfile which deps don’t have hashes?

@happy-kitchen-89482 I wonder if this is https://github.com/pantsbuild/pants/discussions/16943?

Just came here to say that…

📣 You may notice we've started asking that some threads be moved over to GitHub Discussions. The reason is to make sure especially useful conversations can be easily found by others via search engines. Pants is on Slack's free tier, which expires most messages after 90 days. Slack is ideal for casual chat, and we encourage you to keep on posting questions to either Slack or Discussions. If we ask you to move something over to Discussions, don't worry — you didn't post in the "wrong" place! We just felt that particular conversation would be too broadly useful for it to just disappear into the Great Slack Void in a few weeks.

Excited to announce my latest invention! The

external_tool

target! https://github.com/pantsbuild/pants/pull/17277 If a picture is worth 1000 words, an example has to be worth something:

external_tool(
    name="gh",
    source=http_source(
        url="<https://github.com/cli/cli/releases/download/v2.18.0/gh_2.18.0_linux_amd64.tar.gz>",
        sha256="6b091b0b21ee8b0ec257920968905dc065505f5718e5a7de1e9d287320869ef8",
        len=9230158
    ),
    exe="gh_2.18.0_linux_amd64/bin/gh"
)

$ ./pants run //:gh -- --version
gh version 2.18.0 (2022-10-18)
<https://github.com/cli/cli/releases/tag/v2.18.0>

Now you can declare tools in that Pants will take care of downloading and sandboxing, and share those with your org. Or run them as part of other scripts. Have fun 😈

I'm seeing a weird probably-bug after adding a

files(name = "all-the-files", sources = ["**"])

target to the repo root (don't ask me why simple smile). I'm guessing it's because we have some paths in the repository that contain asterisk (don't ask me why simple smile).

InvalidFieldException: The 'source' field in target //some/project/data/[*]/_SUCCESS:../../../../../all-the-files should not include `*` globs, but was set to some/project/data/[*]/_SUCCESS. Instead, use a literal file path (relative to the BUILD file).

The actual path is

some/project/data/*/_SUCCESS

.

Hi, go support is still in beta stage, is there a timeline for the official support?

Hi all, I was wondering if pants had a proper clear-all function to clean out all cached files? I saw that it was supported for older versions, but not the new versions of pants?

How can I set up multiple tool lockfiles that differ for different python “resolves”?

Another probably-bug. It seems like dependency resolution fails when you have import blocks that are formatted like this:

from typing import (

    cast,
)

The relevant code uses

tokenize.generate_tokens

to extract imported symbols on each line of the import, which crashes when a line is empty in multi-line statement. Although arguably this is a bug in the tokenize library itself. How unfortunate.

python infer: I have situation (still debugging) where pyqt5 is used in tests, and there are two PyQT5 libraries (PyQt5 and PyQt5-Qt5) that provide the relevant modules. Leaving Pants to auto-infer

from PyQt5 ...

did not choose the required one,

PyQt5-Qt5

so i added a module Mapping

python_requirements(
    module_mapping={
         "PyQt5-Qt5": ["PyQt5"],
    },

but of course, I need the "other"

PyQt5

library also to be included in the test. Any suggestions to how I resolve this .. I am holding off adding static dependencies for each BUILD file, as there aremore than a handful. is there another technique other than statically adding the two (non-inferred) dependencies ?

What advice would you have for handling dependencies that only exist in conda but not pypi?

I'm trying to understand the purpose/behaviour of

~/.cache/pants/named_caches

. Based on https://www.pantsbuild.org/docs/using-pants-in-ci#directories-to-cache, there's suggestion that it should cache tools/PEXs. However, if I run a CI build with a full cache hit (key = hash of all lock files) on

named_caches

and an empty

lmdb_store

, I still see a lot of lines like the following (plus corresponding

Completed

): For lint/check (ends up being ~20s of wall time spent building the pexes):

Starting: Building isort.pex from isort_default.lock
...
Starting: Building extra_type_stubs.pex
Starting: Building mypy.pex from 3rdparty/python/mypy.lock
Starting: Building 59 requirements for requirements.pex from the 3rdparty/python/default.lock resolve: ...
Starting: Building requirements_venv.pex

For testing (ends up being 2.5 minutes of wall time building pexes):

Starting: Building 42 requirements for requirements.pex from the 3rdparty/python/default.lock resolve: ...
...
Starting: Building pytest_runner.pex
...

I'd be expecting that the ~2GB in

named_caches

would eliminate the 3 minutes of building all these pexes, even if nothing else is cached, but that's apparently not the behaviour. Is there something I may be doing wrong? (A few more details in the thread)

Things have gotten better since I moved everything to a single

python_requirements

target. The one thing that seems to linger though is that certain transitive 3rd party deps aren't picked up. As of now it seems like the

mock

dependency isn't pulling in a transitive dependency on

setuptools

, so tests that use

mock

fail like:

import mock.mock as _mock
/Users/navneeth.jayendran/.cache/pants/named_caches/pex_root/venvs/s/1dd3906e/venv/lib/python3.7/site-packages/mock/mock.py:69: in <module>
    from pbr.version import VersionInfo
/Users/navneeth.jayendran/.cache/pants/named_caches/pex_root/venvs/s/1dd3906e/venv/lib/python3.7/site-packages/pbr/version.py:25: in <module>
    import pkg_resources
E   ModuleNotFoundError: No module named 'pkg_resources'

Manually including

setuptools

in the

dependencies

field of those tests fixes it, although it's not so nice. Also note I'm including

setuptools

in a different

python_requirement

from the main

python_requirements

I'm using, although these two don't overlap at all so there's never no ambiguity problems.

Which version of mock is this @rough-vase-83553?

I suspect it's broken with undeclared deps. That's pretty common with setuptools deps in particular. You can address via: https://www.pantsbuild.org/docs/python-third-party-dependencies#requirements-with-undeclared-dependencies

Is there a way to generate a simple dependenices graph for a given target, when I run

./pants dependencies --transitive path/to/thing

i get a flat list. I am wondering if Pants can show a graph of the dependencies, both within the Repo, but also the 3rd party ones.

Hi all, had another question: does pants support caching individual tests within a pytest target?

Is there any way to get external tool downloads to be retried automatically? We see a regular low-level of flaky failures that I would like to avoid with some automatic retries.

Why does it seem like this step is not cached by Pants? (for a

./pants run ..

on 2.13.0):

59.01s Building 45 requirements for src.python/.../some.pex from the 3rdparty/python/lockfiles/default.txt resolve: Django==3.2.16, JPype1==0.7, ...

As it is rather time consuming (~60s) it’s a bit of a waste to wait at every time I want to run that .pex app.. Asking to see if I’ve missed something obvious, or if it sounds like a bug worth investigating further..

When packaging a python project; is the intention that passing a

pyproject.toml

to

python_distribution

the code isn't packaged if set via dependency. E.g.

dependencies=[
        ":pyproject",
        "//pants-plugins/backends/kubernetes:kubernetes",
    ],

packages no code, while

dependencies=[
        "//pants-plugins/backends/kubernetes:kubernetes",
    ],

properly includes all code. I believe from the website the first variant should work too. It does get traversed while finding deps, but not actually included.

Is there a way to run

pex3 lock create

with the

-r

coming from

stdin

? Coming from

pip-compile

I can

cat requirements.txt | pip-compile -o out.txt -

Hey Pants Team, I'm curious if there's a way to configure linter defaults such as

SkipFlake8Field

. The current way we are doing it is just patching these in a custom plugin but I wonder if there's an easier way to do this.

Does

[python].default_run_goal_use_sandbox = false

allow you to write non-hermetic test cases, e.g. tests on code that reads files directly from the source tree without specifying them as dependencies? It seems like the answer is no but I'm curious how that's the case without sandboxing / hoping there is a way to (temporarily) write non-hermetic tests.

I'm trying to generate API documentation, which requires several steps: 1. Run the Python app to export the schema: create a PEX file, and run it via

experimental_shell_command

capturing the schema as a file 2. Run the doc generator on the schema to create a .html file: install (via NPM) the doc generator and run it on the output of above, all via

experimental_shell_command

3. Package the .html file into an archive:

relocated_files

+

archive

This mostly works, except I'm finding step 1 is... weird. The targets I'm using are, something like:

# path/to/dir/BUILD
pex_binary(name="print-schema", ...)
experimental_shell_command(
    name="schema",
    command="../../../path.to.dir/print-schema.pex > schema.graphql",
    dependencies=[":print-schema"],
    tools=["python3.9", "bash"],
    outputs=["schema.graphql"],
)

Questions: 1. Is it possible to have

experiemental_shell_command

run an in-repo Python file directly, pulling in appropriate internal and external dependencies? It'd be nifty to not need to set up the PEX file if we didn't need to 2. Is

tools=["python3.9", ...]

the right way to pull in Python to run the PEX? I have a feeling that it'll just be just pulling in whatever

python3.9

executable happens to be on PATH (if there is one), rather than the interpreter Pants has gone to great efforts to choose for 'normal' `run`/`test` etc. of Python targets? 3. Is there a better way to refer to the root of the repo/find the dependency pulled into the

experiemental_shell_command

, beyond just

..

the right number of times?

Soliciting opinions on an idea for improved onboarding/developer experience/make-my-life-marginally-easier (which I'm going to build regardless, but just opinions on what we need): We currently have isolated example repos for several backends, but I was thinking of some sort of "kitchen sink" approach, which would be a single monorepo (or a single repo, with submodules 🤷). Not proposing that each backend talks to each other, being completely isolated is just fine - but the idea is showing how a monorepo could be setup. Some benefits: • Non-trivial Pants repo setups can benefit new users (or existing people like me, who forget how config should look sometimes) • For a sophisticated enough repo, plugin development test runs cover a wider array of code (issue for me right now with

pyright

working sometimes, but not other situations) • [Another thing I forgot] Questions: What should the scope of each of these backend examples be (reasonable middleground). • 1st party code • 1st party lib/dep (in another folder/repo) • 3rd party deps • Codegen • ???

Hi all, have some questions on the list of targets that are generated by the pants build. I have a BUILD file for my pytests as such:

python_sources()

python_tests(
    name="tests0",
    sources=["test_a.py"],
)

python_tests(
    name="tests1",
    sources=["test_b.py"],
)

python_tests(
    name="tests2",
    sources=["test_c.py"],
)

and

./pants list

returns the following:

repo/test:tests0
repo/test:tests1
repo/test:tests2
repo/test/test_a.py:test0
repo/test/test_b.py:test1
repo/test/test_c.py:test2

My GHA action to setup pants started failing. Any ideas why? https://github.com/StackStorm/st2/actions/runs/3298749607/jobs/5441375293#step:3:1

Hello. I have a weird issue with

./pants generate-lockfiles

, The command takes forever as it is trying to compile NumPy. I can't find why it can't use a wheel instead of compiling the package and why it tries to install NumPy in the first place. I tried to

pip install -r reqs.txt

on a clean venv to see if something was pulling numpy but nothing appeared. Any idea why numpy appears and how I could debug that?