What would cause an error like the one in the thread? It's happening in our CI system. Haven't been successful reproducing it locally. Clearing out

~/.cache/pants

has not seemed to be helpful.

I blog-posted about some neat hackery we did in CI, if you feel like giving it some love on the orange site: https://news.ycombinator.com/newest

Hi, I don't know if this has been asked before, but I cannot find any solution after searching. I want to add a particular package (prophet) as a requirement for my pants project. In order to be installed, this package requires building (some C++ files need to be compiled), and in the process of generating the

requirements.pex

file, I've been getting the error

ModuleNotFoundError: No module named 'numpy'

. I've added the necessary dependencies following "Requirements with undeclared dependencies" on this page, but I'm still not able to pass this failure. In pants, it seems all the requirements are going to be included in one process, while the package maintainer (see here and here) recommends multiple installing the dependencies first and then in the next stage build and install the package. In light of this, when building

requirements.pex

, is there a way to specify the requirements in multiple stages? Any recommendation on how to make this work?

Hi, community, I made a minimal pants template for Python projects: https://github.com/da-tubi/pants-minimal Hope it will be helpful for newbies!

Is there a faster way to add one new package to requirements and recalculate lockfile than "generate-lockfiles"? Something similar to what tools like poetry or pdm have (e.g. pdm add). The "manually add to requirements.txt and generate-lockfiles" workflow takes very long in our case, and it doesn't change at all during subsequent runs

I ran pant's

HEAD

at the time I'm writing, as follows:

> PANTS_SHA=35ab958651cab95a5342a51247fd41d0e8654439 ./pants --version

with the following diff in my configuration:

@@ -6,10 +6,11 @@ backend_packages = [
   "pants.backend.python.lint.black",
   "pants.backend.python.lint.flake8",
   "pants.backend.python.lint.isort",
+  "pants.backend.python.typecheck.pyright",
   "pants.backend.shell.lint.shellcheck",
 ]
 
-use_deprecated_directory_cli_args_semantics = false
+# use_deprecated_directory_cli_args_semantics = false

but the backend is not found 😕

> PANTS_SHA=35ab958651cab95a5342a51247fd41d0e8654439 ./pants --version
... warnings about Goal.environment_migrated=True ...
Traceback (most recent call last):
  File "/home/churlin/.cache/pants/setup/bootstrap-Linux-x86_64/2.15.0.dev4+git35ab9586_py39/lib/python3.9/site-packages/pants/init/extension_loader.py", line 143, in load_backend
    module = importlib.import_module(backend_module)
  File "/home/churlin/.pyenv/versions/3.9.13/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pants.backend.python.typecheck.pyright'
14:17:41.73 [ERROR] Failed to load the pants.backend.python.typecheck.pyright.register backend: ModuleNotFoundError("No module named 'pants.backend.python.typecheck.pyright'")

@wide-midnight-78598> any clue as to what I'm doing wrong?

In my python monorepo, I have a library at

libs/geometry

with code in

libs/geometry/kaiko/geometry

and

libs/geometry/pyproject.toml

looks like this:

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "kaiko-geometry"
version = "0.0.1"
description = "Kaiko's shared library to help manipulating shape and geometric primitives."
authors = [
    "Axel Lagre <axel@kaiko.ai>"
]
packages = [
  { include = "kaiko" }
]

When I package the library with

./pants package libs/geometry

, I get a wheel whose python code is in directory

libs/geometry/kaiko/geometry

in the wheel. Whereas the code should be in

kaiko/geometry

(because that's the package name). If I

poetry build

instead of

pants

I get the expected behavior. Any clue as to what I am missing in

libs/geometry/BUILD

? Here is its content:

python_requirements(name="kaiko-geometry-reqs") # Used for local sandboxes, dependencies are pinned

poetry_requirements(name="kaiko-geometry-poetry-reqs") # Used for deployment, dependencies are loose

python_sources(
  name="kaiko-geometry",
  sources=["kaiko/geometry/**/*.py"],
  dependencies=[
      ":kaiko-geometry-reqs#numpy",
    ]
)

python_distribution( # <https://www.pantsbuild.org/docs/python-distributions>
  name="kaiko-geometry-dist",
  dependencies=[":kaiko-geometry-poetry-reqs", ":kaiko-geometry"],
  provides=python_artifact(
    name="kaiko-geometry-dist",
    version="0.0.1"
  ),
  sdist = False # only use wheel
)

What's the proper way to set up

resources

with wildcards? The below code doesn't work, but if I list the files individually it does.

resources(
    name="packageA_datafiles",
    sources=["share/jupyter/*"]
)

pex

question — reading through the help, I can’t seem to find any option to do the resolve only without actually producing a

.pex

file on disk (which would download packages from the package index) — some sort of dry run where

pex

could tell me — “yes, I found the packages you are asking for given all the platform constraints you have provided”. See the thread.

Hi, does anyone happen to have a sample Python monorepo that I could take a look at?

Hahaha. I was thinking it would be great to use pyright instead of mypy. So I went looking for the mypy backend, and I stumbled upon a pyright backend too. Thanks @wide-midnight-78598! Now I just have to wait for the next release 🙂

A demo repo for Jupyter Notebooks Integration with Pants: https://github.com/da-tubi/notebooks I tried the pants Jupyter plugin but it is not very easy to use. I think my attempt might be a better solution. Hope Pants developer can make it easier.

Hello, I’m reaching out because bootstrapping pants today caused the whole installation on my macbook pro M1 to break (xcode, git, direnv, virtualenv, kukectl, pyenv’s python...). I managed to get back xcode working by re-setting the path to the command line tools, then git. However it seems all the others are still broken. Is there any explanation or resources about what happens during bootstraping and what is impacted? Looks like most resources are for most somewhat linked with python, maybe that not setting a python-bootstrap in the pants.toml had such really unexpected behaviour?

Is there a formatter for

BUILD

files? I can't find one in the docs 😕

https://github.com/pantsbuild/pants/issues/17199 is an open issue for sharing opportunities to speak at conferences, meetups, podcasts, etc about Pants or related subjects that are of keen interest for the community (e.g. CI/CD, code quality, remote execution, developer productivity, devops, data science tooling, etc). Anyone is welcome to contribute there, and I've already posted a couple opportunities to it.

Is there any way to get pants 1.30 to give us any more information about this message? I've tried the

--print-exception-stacktrace

and

-ldebug

but that doesn't have any specific information about what file it was missing. I've cleared the cache for pants but still no luck

Exception: Error { kind: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" }), paths: [] }

Does Pants somehow bundle its own version of MyPy?

I need some help with transitive transient test failures in a plugin I wrote (this is actually nearly the same code that’s in PR https://github.com/pantsbuild/pants/pull/16932). I created a sample repo that reproduces the problem I’m seeing here: https://github.com/bryanwweber/pants-dependency-tracking What I find is that if I run

./pants test --force ::

repeatedly, it fails about 3/4 of the time. I have no idea why this is occurring… adding as many output and debugging options as I can from the docs (

--print-stacktrace -ldebug --pex-verbosity=9 --keep-sandboxes=always --no-pantsd --no-local-cache

) and comparing logs between successful and failing runs doesn’t turn up anything obvious. The reason I created this test repo is because in our actual code, I can’t even run the tests due to a dependency inference error:

./pants test  ::
15:48:35.48 [ERROR] 1 Exception encountered:

  UnownedDependencyError: Pants cannot infer owners for the following imports in the target pants-plugins/pep621/pep621_requirements_test.py:tests:

  * .pep621_requirements.PEP621RequirementsTargetGenerator (line: 16)
  * .pep621_requirements.rules (line: 16)

If you do not expect an import to be inferrable, add `# pants: no-infer-dep` to the import line. Otherwise, see <https://www.pantsbuild.org/v2.14/docs/troubleshooting#import-errors-and-missing-dependencies> for common problems.

As near I can tell, the configuration between the repos is the same.

So we're using python-grpc (pants supports ootb 👍 ) We're using

pytest-grpc

which we've added to our

[pytest].extra_requirements

in pants.toml... This then means any tests that don't have a transitive dependency on grpc fails. So we've added

grpcio

to our extra_requirements too... This then lead to the two versions of grpcio for the two lockfiles diverging resulting in 'fucky' behaviour in our CICD platform when the old version no longer had a precompiled binary (or our CICD platform changed their environment and was no longer able to compile it) Is there a way to avoid this?

I'm trying to figure out why Pants is giving me memoized test results in some cases when dependencies have clearly changed. What's a good way to investigate this?

Right now Pants doesn't seem to rerun a test even when I modify the test file itself. Adding

--test-force

doesn't make a difference.

I just learned that Pants uses a persistent background process, and killing it helped

How far away is Pants from running natively on Windows? Just not prioritized? Impossible? Works but not supported? I'm considering moving one of our repositories from Poetry to Pants, but it currently supports Windows/Linux/Mac. And some of the those working in the repository aren't super fond of WSL, unfortunately.

I’m curious about how coverage and

--changed-since

interact in testing. Or, rather, I can see that coverage only shows results for the files that actually ran during a

./pants test --changed-since

run. Are there any best practices for using that coverage information in CI? I’d like to use

--changed-since

in our CI to help speed up test runs, but obviously only showing coverage of the files that have been changed doesn’t allow us to gate MRs on increasing overall coverage

Our CI system is getting absolutely trashed with errors like this after upgrading to 2.13.0. Can I please get some assistance on this? It's breaking daily.

10:42:12.54 [INFO] Initialization options changed: reinitializing scheduler...
10:42:13.10 [INFO] Scheduler initialized.
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/buildbot/.pex/unzipped_pexes/7907cee4a80399696d31dbee748702a492b91a80/__main__.py", line 102, in <module>
    from pex.pex_bootstrapper import bootstrap_pex
ModuleNotFoundError: No module named 'pex'

I’m trying to incrementally adopt pants in a project that currently defines all of its dependencies in setup.py (

install_requires

,

extras_require

, etc.) Does pants offer a way to generate

python_requirement

targets for my third party dependencies from the existing setup.py file? Similar to the `python_requirements`/`poetry_requirements` generators?

When I build a distribution, how do I include data files?

Hi all, I wanted to setup remote caching, but not necessarily remote execution for pants. So I tried https://github.com/buchgr/bazel-remote, because I found it very easy to setup, just a single docker command. It's not in the list of tested REAPI servers in the documentation, but I tried it out with my main project (~1k files, ~150kLOC) and it seems to work very well. I did get the following warning a couple of times (not always) on our CI when running

pants export

but it worked correctly, so no worries there.

pants test

does use the remote cache and does not output any warning or error.

+ ./pants export ::
12:21:19.72 [WARN] Failed to read from remote cache (1 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.72 [WARN] Failed to read from remote cache (2 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.72 [WARN] Failed to read from remote cache (4 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.72 [WARN] Failed to read from remote cache (8 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.72 [WARN] Failed to read from remote cache (16 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.72 [WARN] Failed to read from remote cache (32 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.73 [WARN] Failed to read from remote cache (64 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.73 [WARN] Failed to read from remote cache (128 occurrences so far): Unavailable: "Pants client timeout"
12:21:19.83 [WARN] Failed to read from remote cache (256 occurrences so far): Unavailable: "Pants client timeout"
12:21:20.24 [WARN] Failed to read from remote cache (512 occurrences so far): Unavailable: "Pants client timeout"

Two questions: • Are those warnings in any way significant? • Is there something I can do to help add bazel-remote to the list of servers tested with pants? E.g. run a build of a pants-based project and check for success? If something like that can be easily done (without having to setup complex environment, or if they are available as docker images), I'd be willing to do it and give feedback.

Another question: Assuming a mono-repo with multiple first-party packages that have dependencies between them, is there a way to forbid some dependencies and have pants error out if they are detected? E.g. prohibit library

libA

from depending on application

appX

or so? Since pants resolves the dependencies on the file level, it's not bothered by that (or even by circular dependencies), but the library will pull the app when a user installs it, which is what one wanted to avoid in the first place by splitting the code this way.

In a Python codebase, if I have a BUILD file with

python_sources(name="foo")

, is it meant to be possible to do

import foo

somewhere else?