Is there a way to make

pants test

depend on the current package? Normally a

poetry install

would fix this or a

pip install .

but I'm not sure how to fix the module not found error here as I am unsure how to tell pants that my current module is a dependency of the tests

Hey all, getting this error when trying to generate a Python lockfile:

ERROR: Could not find a version that satisfies the requirement pandas==1.5.2
ERROR: No matching distribution found for pandas==1.5.2

When I remove that dependency from my

requirements.txt

the lockfile generation works. This version of pandas also definitely exists: https://pypi.org/project/pandas/#history. I have https://pypi.org/simple/ specified in my indexes field under

[python-repos]

. Also weird is that there are other dependencies that depend on pandas and it is able to locate a much older version, but there's no requirement that should prevent installation of this specific version.

@curved-television-6568: Thank you for

generate-lockfiles --diff

! (I added the option in pants.toml) This is so nice!

Hi all, I'm having a problem with imports not being picked up by Pants, is anyone able to help please? Directory structure:

/src/my_app/my_app.py
/src/my_app/utils.py
/tests/test_my_app.py

my_app.py

from utils import func1, func2, func3

test_my_app.py

from my_app.my_app import function_to_test

pants.toml

[source]
root_patterns = [
  ".",
  "/cdk",   # AWS CDK stuff in Python, not relevant to this
  "/src",
  "/tests",
]

/src/my_app/BUILD

python_sources()

pants dependees src/my_app/utils.py

20:06:56.42 [WARN] Pants cannot infer owners for the following imports in the target src/my_app/my_app.py:

  * utils.func1 (line: 8)
  * utils.func2 (line: 8)
  * utils.func3 (line: 8)

pants dependencies src/my_app:my_app

src/my_app/my_app.py
src/my_app/utils.py

Where can I read more on how the

[test].extra_env_vars = ["VAR1", "VAR2=hardcoded_value"]

works or maybe even the

runtime_dependencies

field? Before my tests run, I want to package/deploy some custom non-python targets using some existing rules I have. And then provide some of the output of the package/deployment to the test (i.e. the location of the packaged file on "the cloud") Could have sworn there was some documentation on it, but I am lost 😞

Using pants, is it possible to build a pex file that contains internal dependencies but no third party dependencies?

if I run

pants lint check ::

it appears wait for flake8 to finish before starting mypy. Why is that? Did I somehow add a dependency from mypy to flake8?

Does anyone have any tips for integrating pants formatting into pre-commit checks? Let's take

black

as an example -- I would like to be able to run

black

with the same config and versions through pre-commit as I do when I execute

pants fmt

so that the results are consistent, but I don't want to config these in two places. So far I've tried: • just run

black

via pre-commit, ignoring pants. This is fairly trivial but it needs me to write a version number in

.pre-commit-config.yaml

as well as in

pants.toml

, and that's slightly annoying because those can get out of sync • change pre-commit to run

plz fmt

instead of using the

black

plugin directly. This avoids the need to duplicate config, but it has the downside that now pre-commit doesn't format build files or things that pants doesn't "know about", like stuff under

3rdparty/

. Any pointers from someone who has trodden this path before? Same question applies to lint and check.

I'm having a couple Python dependency issues when running

pants test ::

that I'm struggling to resolve. Most of my tests pass fine but a handful are failing due to missing dependencies. I'm also getting complaints that pants cannot infer an owner for

pytest

- for this repo (and all of our existing repos) we maintain a regular

requirements.txt

and then a separate list that also includes dependencies necessary for local testing. Is it possible to set up something similar in pants? I have

extra_requirements

and a

lockfile

specified under my

[pytest]

header in my

pants.toml

.

Just want to say the series of green checkmarks after this felt deeply satisfying:

pants experimental-deploy --filter-address-regex='\-dev$' src::

Okay I've resolved most of my dependency inference issues I was having earlier but I'm still getting this one:

from azure.cosmos.exceptions import CosmosHttpResponseError
E   ModuleNotFoundError: No module named 'azure.cosmos'

Does this have to do with the module mapping maybe? Even though I see azure.cosmos in the default mapping.

what's the best pattern for avoiding accidental dependencies on test code from production code? Right now I have a single lockfile for all my code, but it includes things like

hamcrest

. I get that the dependency won't be packaged when I wrap up my production code, but it feels like it's very easy for me to accidentally introduce dependencies on the "wrong" things in prod code. The obvious idiom seems to be that I would create a second resolve and move the test stuff there, but then I suddenly have to parametrise lots and lots of things because I need production code to be usable in both resolves and test-only dependencies need to become more verbose too. Is there perhaps a way to "layer" resolves so that my test one "extends from" my production one, and pick different resolves by default for prod vs test?

https://www.pantsbuild.org/docs/tagging-docker-images "We are looking into making some common dynamic data, such as the git sha, automatically available in the core Docker plugin in the future." Is there a ticket that maps to this? I'm tentatively thinking it would also be useful for the helm backend to dynamically set the version on publish.

So someone was reviewing a macro I wrote and asked something along the lines of "do all docker_image targets have `name`s?" Neither

pants help-advanced docker_image

nor

pants help-advanced target

mention a

name

field. But I'm pretty sure targets have names? From the code it looks like the "answer" is: • (A) All `pants.engine.target.Target`s have a name, but it maybe an implicit one. • (B) but! In the case of macros, when you are looking at kwargs said implicit name would not yet be available. So I think I've answered my own question. But should I have been able to answer (A) from

help

and not grepping the pants source code?

If I set a package manually in `pyproject.toml`(like

mypy = "1.1.1"

), will the

[mypy].version

config follow the one installed? Because then I could use renovatebot to keep the version of this tool maintained and updated.

Building on the thread above but a slightly different problem: I now have mypy 1.1.1 configured correctly. When I do

pants check my_file.py

mypy raises things that I expect - great! I now configured a pre-commit hook as such:

repos:

- repo: local

hooks:

- id: check

name: check

entry: pants check

language: system

files: ^.*.(py|pyi)$

When I now git add and git commit the same file mypy fails with this error:

.cache/pex_root/venvs/5cb0b2dcd2357da2bbde0d877bde9b817c555a3e/fde49d60105c0ef9833901ddf249f36d7deded9a/lib/python3.11/site-packages/numpy/__init__.pyi641 error: Positional-only parameters are only supported in Python 3.8 and greater

This problem comes from an old mypy version and should be already fixed (ref). I am puzzled where this discrepancy of different mypy version stems from and would be excited if someone could help me out here!

Related to the issues I had yesterday, has anyone seen these resolve errors before?

09:30:50.92 [ERROR] 1 Exception encountered:

Engine traceback:
  in `lint` goal

ResolveError: The file or directory 'e' does not exist on disk in the workspace, so the address 'e' from the `dependencies` field from the target 3rdparty/python:globals#gql cannot be resolved.

Potentially dumb question but how do I only regenerate lockfiles for resolves that have been updated?

I try to run a single test file but pants does nothing

pants test path/to/test.py

Does anyone have an example where

docker_image

is an dependency? Meaning that some other targets depend on a docker image. e.g. running tests using that image, building another image using it as a base image, etc.

can anyone point me to info on how (whether) transitive dependencies resolved and stored in pants lockfiles end up with version constraints in packaged artifacts? My instinct is that if I use

generate-lockfiles

to create a lockfile and then

pants package

some code, the resulting artifact should demand that transitive dependencies are frozen at the versions resolved in my lockfile -- because that's what would make consuming that output downstream repeatable based on what was built (and tested). That doesn't seem to be what's happening, and I don't know whether there's some

pants

config that would make it be true, or whether I'm looking in the wrong place, or indeed -- given that I am not well-versed in the python ecosystem -- whether I am wanting something unreasonable or unpythonic. Example of what I'm doing in the thread so I don't make the chat too noisy 🧵

I am trying to update to pants 2.15.0 and even with

skip_mypy=True

set in

___defaults___

, pants check is running mypy. It was working with 2.14 though i.e. skipping mypy check correctly.

Is there a way to batch tests but still have the same number of "reassuring green checkmarks"? ✅ That is instead of:

✓ test/unit/modules/test_critical.py:py310 and 16 other files succeeded in 22.74s.

Get 16 green checkmarks just as you would without the batching?

I'm trying to set

named_caches_dir

and

local_store_dir

in my

pants.ci.toml

. I have

PANTS_CONFIG_FILES=pants.ci.toml

set an environment variable in my CI, but when I look at

./pants help-advanced global

(again in my CI) I see

local_store_dir
      default: /root/.cache/pants/lmdb_store
      current value: /root/.cache/pants/lmdb_store

In my

pants.ci.toml

I have

[GLOBAL]
pantsd = true
named_caches_dir = "/codefresh/volume/.cache/pants/named_caches"
local_store_dir = "/codefresh/volume/.cache/pants/lmdb_store"

In

pants.toml

I have

[GLOBAL]
pants_version = "2.14.1"

I can sanity check

root@codefresh:/codefresh/volume/archipelago$ echo $PANTS_CONFIG_FILES
pants.ci.toml

and

ls /codefresh/volume/.cache/pants

only shows

setup

. If I run

./pants test capstan/:: --local-store-dir=/codefresh/volume/.cache/pants/lmdb_store

I see

lmdb_store

. when I

ls

. Am I doing something wrong, I'm not sure why my configuration isn't getting picked up

i have a scenario where a pytest works, but within pants, the 3rd party library throws quite an obscure error. thinking it may be a module that is missing, I added all

//:reqs#..

to the test, but that did not resolve it. what is the best way to debug this issue (I have 15 test cases that fail for -not same- reasons, so I am after the technique suggestions.

A strange migration tip; situation. 1. have an existing sizeable python repo with embedded /test/folders - hundreds of tests (makes sense, tests are local to the code) - large++ code base 2. some test folders have modules, (helpers, utils) but are missing

__init__.py

files goal: locate all directories under test folders, that don't have

__init__.py

files, but do have

*.py

files, this is not fool-proof but it did locate the > 30 folders I had to investigate ; to remove the "pants can't infer" (output of a test, was mostly written to a

output/

subfolder (.gitignore'd) ) • read as, find directories that are "test" and have

.py

files, but no

__init__.py

files

❯ find . -type d -wholename '*/test*' -not -name __pycache__ -not -wholename '*/test/*output*' -not -wholename '*/.venv*' -exec find {} -maxdepth 1 -type f -name '*.py' -not -name 'test_*' -quit \; -not -exec test -e "{}/__init__.py"  \;  -print

coverage-py: is there something special I need to do, to generate the coverage reports via pants ? I have this

[coverage-py]
report = [ "html", "console" ]

in

pants.toml

and I tried having both no config in

pyproject.toml

and added a few basic examples. I am expecting a report generated to the dist folder

--coverage-py-output-dir=<str>
  PANTS_COVERAGE_PY_OUTPUT_DIR
  output_dir
      default: {distdir}/coverage/python
      current value: {distdir}/coverage/python

I am just running

pants test ::

and expeciting reports 🙂 do I have it wrong ?

Is there some method for branching on pants version in BUILD files? I've done the following as a macro:

def pants_at_least(version: str) -> bool:
    pants = __import__("pants")

    return pants.version.PANTS_SEMVER >= pants.version.Version(version)

But it's clearly wrong since I have to "force" the import to avoid the guard rails. My end-goal is to enable early adoption/testing of future pants versions in CI. Below is an instance from my open OCI backend where this would be useful in my examples directory:

empty_base = "//:empty"  # see [oci].empty_image_target

# using pants 2.15+ there's an built-in for empty bases, but before that we need to declare an empty image to use:
if not pants_at_least("2.15.0.dev0"):
    oci_image_empty(
        name="empty",
    )
    empty_base = ":empty"

oci_image_build(
    name="with_empty_base",
    base=[empty_base],
    packages=[":example"],
    tag="latest",
)

I almost certainly missed it, but how does one invoke pants on the commandline? That is to say

pants test ::

What does

::

represent?

Hitting an error with

poetry_requirements

for everybody's favourite package: Torch. In pyproject.toml I have the following

[tool.poetry.group.ml.dependencies]
torch = [
    {markers = "platform_machine == 'arm64' and sys_platform == 'darwin'", url = "<https://download.pytorch.org/>..."},
    {markers = "platform_machine == 'x86_64' and sys_platform == 'darwin'", url = "<https://download.pytorch.org/>..."},
    {markers = "platform_machine == 'x86_64' and sys_platform == 'linux'", url = "<https://download.pytorch.org/>..."}
]

Either this is borked somehow, or there's a bug with parsing, because I'm receiving an error:

ValueError: Invalid requirement 'torch @ <https://download.pytorch.org/whl/cpu/torch-1.9.1-cp39-none-macosx_11_0_arm64.whl;(platform_machine> == 'arm64' and sys_platform == 'darwin')': 

Parse error at ""== 'arm6"": Expected string_end

The quoting in the message looks kinda suss. I'm unsure whether I can rewrite these into a form that pants will prefer, or whether I should remove them from pyproject and do something else altogether.