We upgraded from 2.6 to 2.13 and have seen memory usage of pants go up sometimes 3-4 times. This is happening even for

list

, but also for

lint

(that’s the one I saw running out of memory, but I can look into it more). I cannot bisect to find where this changed as most of the upgrade in terms of getting list to work was done in a couple of commits. What could I do to look into this?

bumping the name poll for visibility: https://pantsbuild.slack.com/archives/C046T6T9U/p1661516200017449

One question, I have the following problem. I am not sure the best way to solve it so maybe someone has an idea. I have created a docker image, the main app of this image is packaged as pex. However, the environment where it will run only has a python interpreter different from my local version, this causes my packaged version to fail. Is there a way to create my pex so that it can be run in a different interpreter from where I compile the pex?

Another question 🙂 For some reason a specific version of a library does not appear in the output of ./pants --dependencies-transitive dependencies /somepath .... the library is added to the pool of available libraries as others that do show in the output. Why can this happen?

Does pants acknowledge pytest subtests? I have a use case where outside of pants pytest shows the subtests and executes them all even if one test fails. However if those same tests are run in pants, pants only executes until one fails then aborts the execution. I'm on pants 1.30 currently, but am curious if 1.30 supports subtests, or if subtests may be recognized in newer versions of pants

I am trying to add

mypy

to our project by following https://www.pantsbuild.org/docs/python-check-goal . However, whenever I run

./pants check ::

, it fails to find any third party library from our

requirements.txt

, e.g.,:

File "/tmp/pants-sandbox-WbBUY8/.cache/pex_root/venvs/6633c908304aa46390e473de2960e553ff125f02/108a3ddc84230ab282ea6312e06cb68f51008ce5/pex", line 244, in <module>
    sys.exit(func())
File "/home/irvan/.cache/pants/named_caches/pex_root/venvs/s/8470873e/venv/lib/python3.9/site-packages/mypy/__main__.py", line 12, in console_entry

...

File "/tmp/pants-sandbox-WbBUY8/apps/solvent/celery/celery.py", line 1, in <module>
    from celery import Celery

ModuleNotFoundError: No module named 'celery'

The

requirements.txt

has the celery library:

celery==5.2.7

I've been trying to debug this to no avail. This is what I tried so far: I noticed that the python virtual environment in the trace above (

/home/irvan/.cache/pants/named_caches/pex_root/venvs/s/8470873e/venv/lib/python3.9/site-packages/

) does not have celery installed. I tried the

extra_requirements

field mentioned here , but it does not seem to accept a filename, e.g.,

requirements.txt

. I could paste the entire content of

requirements.txt

into that field (This actually works from what I tried), but this seems very hacky. In case it helps, this is the current

mypy

settings in

pants.toml

. I am following the settings from the example django project.

[mypy]
lockfile = "lockfiles/mypy.txt"
version = "mypy>=0.961"
interpreter_constraints = [">=3.9"]
extra_requirements.add = [
  # This is a mypy plugin, as well as a type stubs repository. So it must be mentioned
  # here as well as in requirements.txt.
  "django-stubs>=1.1,<2",
]

Thank you very much!

Hmm, that should not be happening… Can you post a small dummy repo that reproduces this?

It’s much easier to debug when we have an example of it happening that we can reproduce

Hello! I'm brand new to Pants, but am so far finding it very intuitive (so thank you devs! 👏). I'm running into an issue with a specific Python requirement, though:

morecantile

(found on Github here). When I try to do

./pants generate-lockfiles

, I get an error like this:

20:25:25.78 [INFO] Completed: Generate lockfile for python-default
20:25:25.78 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:

stderr:
WARNING: Discarding <https://files.pythonhosted.org/packages/82/9e/718d2802f64a17b16a0589a97fc3e454f013400403672acde445d289b271/morecantile-3.1.2.tar.gz#sha256=72aacd46a8e74032d081befe2e76ff0861abc71ae95940be84580202be474ad6> (from <https://pypi.org/simple/morecantile/>) (requires-python:>=3.7). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement morecantile==3.1.2
ERROR: No matching distribution found for morecantile==3.1.2
pid 2973543 -> /home/ubuntu/.cache/pants/named_caches/pex_root/venvs/05babdc9287e6fa5ed263c085edf8c6d27209bb3/ddab8011daaee380698ac2fb9701af18c90c03f6/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/ubuntu/.cache/pants/named_caches/pex_root --log /tmp/process-executionf2VqiP/.tmp/pex-pip-logwme1obv6/pip.log download --dest /tmp/process-executionf2VqiP/.tmp/tmpkbbqnlo4/usr.bin.python3.8 --requirement __pip_args.txt morecantile==3.1.2 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None

It seems that I can generate lockfiles containing versions of

morecantile

up to and including

3.0.5

. After some light investigation, I suspect that this is because starting at the next version, the distribution's version argument in the

setup()

function is removed (see here). I'm guessing they're determining this dynamically during build. Which, might be a problem in itself. But, Poetry seems to be able to install this package fine. And so can Pants if I install from Github directly (instead of PyPI, ie. by specifying

git+<https://github.com/developmentseed/morecantile@master#egg=morecantile>

in

requirements.txt

. Does the lack of a version in the

setup

function seem plausible as the source of the error above? If yes, is this just Pants being pickier than eg. Poetry?

Hi! What are your repo’s interpreter constraints?

I can reproduce this with pex alone:

python -m pex.cli lock create  --interpreter-constraint=">=3.8,<3.9" --resolver-version=pip-2020-resolver morecantile==3.1.2

vs

python3.9 -m pex.cli lock create  --interpreter-constraint=">=3.8,<3.9" --resolver-version=pip-2020-resolver morecantile==3.0.5

I finished contributor setup and following is a snapshot of a portion of the logs:

Compiling mock v0.0.1 (/root/pants/src/rust/engine/testutil/mock)
Compiling process_execution v0.0.1 (/root/pants/src/rust/engine/process_execution)
  Finished release [optimized + debuginfo] target(s) in 14m 38s
fatal: No names found, cannot describe anything.
05:04:31.99 [INFO] Starting: Resolving plugins: hdrhistogram, toolchain.pants.plugin==0.22.0
05:04:55.91 [INFO] Completed: Resolving plugins: hdrhistogram, toolchain.pants.plugin==0.22.0
05:05:01.28 [INFO] Initializing scheduler...
05:05:02.03 [INFO] Scheduler initialized.
No goals specified.
Use `./pants help` to get help.
Use `./pants help goals` to list goals.
(pants-dev) [root@9a5752857756 pants]#

What is this

fatal: No names found, cannot describe anything.

in the logs? Does it mean that there was no fatal error since no names were found?

I've finally found that why many open source contributors to Backend.AI has suffered from the pants installation: https://github.com/lablup/backend.ai/issues/690 If we could have macOS + arm64 wheel for Python 3.8.9 (the system Python shipped with macOS Monterey), the life would become much easier. 😉

Hello everyone, I just wanted to know if pants supports merging xunit reports?

I notice that each daemon restart re-runs the relevant `experimental_shell_command`each time. We're downloading from network, so this can be expensive. Any pointers on debugging what might've changed in the key?

I just read this article from Proton - I wanted to quote dope snippets from it, but I'd basically be copying/pasting the whole article. I've already forwarded it to a few clients where we're discussing pros/cons of polyrepo vs monorepo. https://proton.me/blog/engineering-polyrepo-monorepo

Hi, I am getting this strange error after deploying my django proj on lambda. My lambda fn url - https://gknygoefhm3vdp4k7yso7mqudy0wuzaj.lambda-url.ap-south-1.on.aws I have just endpoint setup 'test/' and that works fine locally. Link to the repo - https://github.com/reksus/tarun-debug

What’s the correct path for copying a

pex_binary

inside docker? I tried the

<http://path.to|path.to>.target/target.pex

but doesn’t work 🧵

Hello, I have a small problem related to where git is installed. One of my colleagues has his git executable installed under /opt/git/bin and pants complains that it can't find the git executable. The list of search paths shown on the error message doesn't contain /opt/git/bin even if set in the PATH env variable of the shell. Is there an option like [python-bootstrap].search_path for git?

I think I read something about making MyPy faster in this channel. Am I right and if so, is it something I can use with pants 2.12 ?

Hi. I’m trying to package a new app as a pex_binary and for the first time I’ve run into a problem that I can’t solve using just the docs. The app itself is in python, and has 32 dependencies, some of substantial size (e.g PyTorch). When I try to run

pants package

on the target, after around 10 minutes it crashes with “Failed to digest inputs”. I know this isn’t the most extensive description, but I’m not sure what and where I’d have to share for someone to help me. I’d be very grateful for any kind of help.

level: Info,
    append_only_caches: {
        CacheName(
            "pex_root",
        ): RelativePath(
            ".cache/pex_root",
        ),
    },
    jdk_home: None,
    platform_constraint: Some(
        Linux_x86_64,
    ),
    cache_scope: Successful,
}

Failed to digest inputs: "Error storing Digest { hash: Fingerprint<3057fd045aa4878c7e691f962d4f56de483a2c461b2c6066fa94a67e0a32bd78>, size_bytes: 2663814611 }: Input/output error"

Is there another

grpc-python-plugin

version that we can use? The default one is

1.32.0

which requires a

GCC

version we don't have. https://www.pantsbuild.org/docs/reference-grpc-python-plugin

I’m still working on my Python packaging, and I’d like to use calver (

YYYY.M.D

) to version my packages, instead of semver. Additionally, if this is not a release build, I would like to add the most recent git commit hash to the version, so something like

2022.8.30+gabcd123

. I see that there’s a

vcs_version

target, which uses

setuptools_scm

to pull the most recent git tag, but that doesn’t quite fit what I want to do. Is there any existing code out there to solve this? If not, I started going about this as a plugin, but I’m stuck because I don’t see how the

vcs_version

target knows that it needs to call

setuptools_scm

. I suppose the relationship is that the

generate_python_from_setuptools_scm

rule depends on a

GenerateSourcesRequest

which has the

VCSVersionDummySourceField

as its input, and the

vcs_version

target has

VCSVersionDummySourceField

as one of its required fields. What I’m not clear about is whether I should create a new field specifically for my own VCS-ish version, or if I can reuse the existing fields inside pants?

Seeing some weird exit code when running

./pants test

on circleci:

18:57:59.48 [ERROR] Completed: Run Pytest - src/api/views/test_mfa.py:tests failed (exit code -9).

Is this an OOM error? I would have thought we wouldn’t hit it since we are using a 4core circleci docker executor with 8GB ram (and I haven’t changed the pants resource limits at all)

Hi guys. We are trying to deploy a Lambda Docker image, but keep getting this error. Any ideas on how to fix?

{Error=TypeError, Cause={"errorMessage": "the 'package' argument is required to perform a relative import for '.app.lambdex_handler'", "errorType": "TypeError", "requestId": "", "stackTrace": ["  File \"/var/lang/lib/python3.9/importlib/__init__.py\", line 122, in import_module\n    raise TypeError(msg.format(name))\n"]}}

Is lambdex not compatible with python 3.9?

Hi, I am getting following error while running

./pants package

goal in my CI:

pex.result.ResultError: There were 2 errors downloading required artifacts:
1. setuptools 57.5 from <https://files.pythonhosted.org/packages/4b/b9/71687c5d2034c863db1db2e0704f5e27581ff3cb44d7f293968c5e08ceb3/setuptools-57.5.0-py3-none-any.whl>
    <urlopen error [Errno 97] Address family not supported by protocol>
2. wheel 0.37.1 from <https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl>
    <urlopen error [Errno 97] Address family not supported by protocol>

This is most likely due to restricted access but I have

indexes

(not

indexes.add

) configured under

[python-repos]

subsystem in my

pants.toml

. Is there anything else that I can do?

Hello guys, would it be possible for pants to use the

grpc_python_plugin

from the bin of the grpc installation rather than using a standalone binary? Just wanted to understand why it is not done that way currently before I open a feature request.

Just an update on this. We downgraded the docker image to python 3.8 and it fixed the issue. We may want tell people that lamdex is incompatible with python 3.9.

Is there a way to check what

pants.toml

options are in effect during a particular run of Pants e.g. if I am running

./pants package

, I would like to see what

python-repos

options are active.