I exported a virtual environment in my devcontainer with pants and it successfully created a distribution with all necessary modules. However the imports fail when coding. Isn't it enough to add

"python.defaultInterpreterPath": "./dist/export/python/virtualenvs/default/3.8.13/bin/python",

in my vscode devcontainer settings? When I add

"python.analysis.extraPaths": ["./dist/export/python/virtualenvs/default/3.8.13/bin/python"],

analysis with Pylance starts working, but the imports still fail.

Is there an existing good pre-commit config for a

./pants tailor

check that can auto-fix missing BUILD files?

Is there any way to add some kind of template for the

instructions

field of

docker_image

and then reuse it for multiple applications? E.g.

instructions=[
    "FROM python:${VARIANT}"
    "COPY ${DYNAMIC_DIR} /app",
  ]

Background: the Dockerfile for a lot of services looks almost the same.

Hi. I am trying to pipeline

docker_image

with

pex_binary

on M1 . docker image created but when i try to run it i get

Failed to find compatible interpreter on path /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.

Examined the following interpreters:
1.) /usr/local/bin/python3.9 CPython==3.9.13

No interpreter compatible with the requested constraints was found:

  A distribution for pandas could not be resolved for /usr/local/bin/python3.9.
  Found 1 distribution for pandas that do not apply:
  1.) The wheel tags for pandas 1.4.3 are cp39-cp39-macosx_11_0_arm64 which do not match the supported tags of /usr/local/bin/python3.9:
  cp39-cp39-manylinux_2_31_aarch64
  ... 367 more ...

As I understand pex and docker image was built for different architectures. How can I solve it?

Hi, is it possible, that

__defaults__

gets ignored somehow by BUILD files inside a subdirectory? Maybe this is a bug, because I have a top level

django/BUILD

file which contains:

__defaults__({
    (python_tests): dict(dependencies=["django/django_core/settings.py"])
})

And a BUILD file under

django/sub/tests/classes/BUILD

which just defines:

python_tests(
    name="tests",
)

The dependencies goal in the subdirectory does not reference

django/django_core/settings.py

If I extend the contents of

django/sub/tests/classes/BUILD

the dependency

django/django_core/settings.py

gets shown correctly:

__defaults__({
    (python_tests): dict(dependencies=["django/django_core/settings.py"])
})
python_tests(
    name="tests",
)

Running into a weird issue with Python code coverage. I'm using Pytest and

pytest-cov

, and am trying to refactor a bit of code to use Python's "walrus operator", introduced in Python 3.8. I'm using Python 3.8.13, my syntax is correct, my tests run and pass, but the coverage reporting fails with the following:

10:40:26.59 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Generate Pytest xml coverage report.' failed with exit code 1.
stdout:
Couldn't parse '/tmp/process-executionhBLEdi/build-support/packer_hcl.py' as Python source: 'invalid syntax' at line 23

stderr:

This line is where the walrus operator was introduced. I'm using

pytest-cov

3.0.0, which supports Python 3.10. I see no issues or PRs about this in either Pytest or `pytest-cov`'s repositories. Is it possible that something in how Pants is running things is causing this? Has anybody else seen anything like this? Thanks in advance.

I'll be live on the Code Sandwich Hour in 30 minutes, talking about Pants and, presumably, sandwiches: https://cfe.dev/talkshows/july2022-benjy-weinberger/

Hey there, I've got two questions: 1. What controls the python version used by pants itself? I thought it was the

[python-bootstrap]

config, but specifying ["<PYENV>"] isn't working (which may be a pyenv/path issue) 2. What's blocking pants from using python 3.10 (I see the tracking ticket https://github.com/pantsbuild/pants/issues/14111). I see that pex supports python 3.10 as of Novemberish per https://github.com/pantsbuild/pex/issues/1487.

Hi, informal poll: how would folks feel about no longer releasing Pants with Python 3.7 in Pants 2.14+ and instead adding 3.10? This is only for running Pants itself - does not impact your own code (feel free to comment)

Another poll for a new policy: Pants will release for the most recent Python version + 2 before. So, Pants 2.14 will be 3.8-3.10. When Python 3.11 is released, the corresponding Pants release series will be 3.9-3.11 (Pants release series will support their interpreter range indefinitely. Meaning, Pants 2.13 will always be 3.7-3.9, for example)

Planning to use pants to build docker images using pex as a "build environment" in a multi-stage build (we use python:3.10-slim + lambdaric). The images are deployed via aws cdk stacks.

Hello! I’m thinking of trying out pants for a hackathon, for a large scala codebase. Going to be asking lots of probably basic questions, thanks in advance!

First question: how can I tell pants to ignore BUILD files?

Hey, i've been using pants for a while and i'd like to use spring boot actuator for environment infos, in particular also the git commit hash of the build. i really don't want to use git hooks for this to avoid ongoing merge conflicts and also would like to avoid fishy scripts in my CI pipeline that do some arbitrary bash magic. my preferred solution would be to include this somehow in the pants package goal, where the commit hash is automatically derived during packaging and that's it. my idea would be to use a resource as a dependency of my pex target, generated by a dependency on experimental_shell_command that pushes the commit hash in the resource file. Not entirely sure whether this actually works, but was wondering if someone can suggest a better solution or if there's some out of the box solution i'm not aware of yet. cheers!

Running into a weird issue with Pants which worked before. It’s unable to build requirements.pex. Failing with a requirement parsing error. Details in 🧵

My 2¢ on the scala setup, I know it’s fairly new but very explicitly listing out an example jvm_artifact and

[scala].version_for_resolve

to get some example working (with default and non-default version) would have been helpful. I was able to figure it out with some trial/error though. Also, it’s quite possible I missed it, but wasn’t able to find a page discussing

resolve

in more detail - it seems like it’s an important pants concept.

Is there some tool/hacky way to convert a pom.xml to the jvm_artifacts that pants requires?

Hey! If I have several applications that run unit-test against third-party containers, can I run as many docker-composes? It feels like the only option, when it comes to running containers, is to merge all my docker-composes into one, spin it up first and then run pants, somehow dealing with parallelism. And how can I skip docker-compose if there are no changes to apps which unit-tests rely on services in docker-compose?

Hey, when using

pants test ::

for testing django the performance is much worse than just using

pytest

. Each test takes 8.5-9.5 secs (constantly) , because of some pants setup I think. Using pytest most of the tests run in under one second. I thought pants could run tests in parallel but only one core out of 4 is used. Maybe I am doing something wrong. Also caching seems not to work: When rerunning tests (without changing any files inbetween) every tests gets called again, not only the ones that failed. Is this because I am using a development version of pants or a configuration error? PANTS_SHA: f721c1077f97ef1aacba2a093f3f5cfd9677dc75

[pytest]
lockfile = "lockfiles/python/tools/pytest"
version = "pytest>=7.1.2,<7.2"
extra_requirements.add = [
  "pytest-django==4.5.2",
  "pytest-icdiff==0.5",
  "mixer==7.2.1"
]

Hi there! I'm quite new to Pants so maybe I'm missing something but... On

pytest

you can use markers to run tests selectively. So maybe you want to 1st run all unit tests (quick, no services, etc). Then run tests that require a database and some local services. Then run tests that require external services/APIs. The idea being to not waste time testing some API client if some unit test will fail after it. The "normal" way is to use markers and run pytest one time for each. Another way is to sort all tests (across all modules) to run unit first, integration after. But I cannot see a way to leverage this in Pants (without messing around with environment variables). What is the "pants way" to run unit & integrations tests in separate goals?

I'm trying to figure out an import error, I'm having problems importing a module during a test.

tests/test_hyperspec.py F                                                [100%]

=================================== FAILURES ===================================
_________________________________ test_import __________________________________

    def test_import():
>       import sphinxcontrib.hyperspec
E       ModuleNotFoundError: No module named 'sphinxcontrib.hyperspec'

tests/test_hyperspec.py:19: ModuleNotFoundError
- generated xml file: /tmp/process-execution1Z9sbF/tests.test_hyperspec.py.tests0.xml -
=========================== short test summary info ============================
FAILED tests/test_hyperspec.py::test_import - ModuleNotFoundError: No module ...
============================== 1 failed in 0.08s ===============================

I can see the files there, but i can't seem to actually import the module

$ find sphinxcontrib tests 
sphinxcontrib
sphinxcontrib/cldomain.lisp
sphinxcontrib/cldomain.py
sphinxcontrib/test.lisp
sphinxcontrib/__init__.py
sphinxcontrib/package.lisp
sphinxcontrib/hyperspec.py
sphinxcontrib/version.lisp-expr
tests
tests/test_cldomain.py
tests/__init__.py
tests/__pycache__
tests/__pycache__/__init__.cpython-38.pyc
tests/__pycache__/test_cldomain.cpython-38-pytest-7.1.2.pyc

I went into the directory directly and tested with a local python and with the sandbox python, but the cached one can't import stuff? I commented out the contents of the files completely, to make sure it wasn't an issue with loading them

$ cd /tmp/.tmpDGxM5D 
$ python
Python 3.8.5 (default, Aug 13 2020, 15:51:26) 
[GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sphinxcontrib
>>> import sphinxcontrib.cldomain
>>> 
 
$ /home/russell/.cache/pants/named_caches/pex_root/venvs/s/a67a3153/venv/bin/python3.8
Python 3.8.5 (default, Aug 13 2020, 15:51:26) 
[GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sphinxcontrib
>>> import sphinxcontrib.cldomain
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'sphinxcontrib.cldomain'
>>>

any idea what could be going on? i'm on pants 2.12.0

Hey folks, I just submitted a bug report on Github that'd I'd love to get people's thoughts on. Not sure if this is an actual bug or if I'm doing something silly here. Looking forward to working with you all on this! I'll be AFK in about 30 minutes and will be away until tomorrow sometime (or perhaps all the way until Monday), but I'll be responsive on this when I get back to a computer

Hi, I am trying to create a plugin which will trigger a homegrown system to create venvs for libraries (inside libraries' root folder) in the monorepo. I am going about creating a

Process

and

await Get

the

ProcessResult

but the plugin is ending in following error:

Error launching process: Os { code: 2, kind: NotFound, message: "No such file or directory" }

. I am also having a hard time in providing a workspace folder location to the Process. Could anyone please help me with the above error and providing a workspace folder?

hi folks, as some of you know, PyPI has designated the top 1% of Python projects (by number of downloads) as critical projects, whose maintainers must use 2FA in order to publish. It turns out that both Pants and PEX are in that designation! Thanks to you all for using and contributing to Pants and PEX, and helping grow this community. Onwards and upwards!

I'm working on alternative plugin for deploy_jar, where we merge all first party jars into one jar (and strip timestamp, for reproducibility) and leave the third party as is. How can we tell if

ClasspathEntry

is third party or first? I thought about adding a

third_party

flag to

ClasspathEntry

but it seems that a

ClasspathEntry

can actually hold both first party and third party jars. One alternative is to use some kind of regex to recognize coursier jars, but I'm not a big fan of it. Few reasons for the plugin: 1. Merging all jars can take time 2. we want reproducible jars (no timestamps or metadata), coursier is already reproducible, so we only need to strip first party jars. 3. From our experience, merging jars (fat jar) can actually override resources in un-determinstic way. 4. We would later pack everything with Kaniko and only push the image to registry if digest is different.

I'm stuck again this time at a new error,

pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/russell/.cache/pants/named_caches/pex_root/unzipped_pexes/65af0cca88bb416d7230e8e933478380260e9e31.
Needed cp39-cp39-manylinux_2_33_x86_64 compatible dependencies for:
 1: Sphinx
    Required by:
      sphinxcontrib-cldomain 0.14
    But this pex had no ProjectName(raw='Sphinx', normalized='sphinx') distributions.
 2: setuptools
    Required by:
      sphinxcontrib-cldomain 0.14
    But this pex had no ProjectName(raw='setuptools', normalized='setuptools') distributions.

I have verified that my python_distribution correctly generates with dependencies on these 2 libraries in it's metadata, and the libraries show up as dependencies

$ ./pants dependencies --transitive tests:tests0
//:pyproject
//:sphinxcontrib-cldomain
//LICENSE:cldomain_files
//README.rst:cldomain_files
//src/sphinxcontrib/__init__.py:../../lib
//src/sphinxcontrib/cldomain/__init__.py:../../../lib
//src/sphinxcontrib/cldomain/cldomain.py:../../../lib
//src/sphinxcontrib/hyperspec/__init__.py:../../../lib
//src/sphinxcontrib/hyperspec/hyperspec.py:../../../lib
3rdparty/python#Sphinx
3rdparty/python#setuptools
3rdparty/python/requirements.txt
src/sphinxcontrib/cldomain/cldomain.lisp:lisp_lib
src/sphinxcontrib/cldomain/package.lisp:lisp_lib
src/sphinxcontrib/cldomain/test.lisp:lisp_lib
src/sphinxcontrib/cldomain/version.lisp-expr:lisp_lib
tests/test_cldomain.py:tests0
tests/test_hyperspec.py:tests0

but i get the

ResolveError

as above when i run

./pants tests tests:tests0

the code i'm working off is https://github.com/russell/sphinxcontrib-cldomain/tree/013c10294fb63c5371cbef92ff6999a9845aee32

hello again friends 👋 I have some tests I wish to run against a database and I would like to use a docker container when those tests are run to serve it. Now I can pretty simply do this by just running it first and then running

./pants test

on the desired target(s), but I'm really digging using

pants

as my do-all tool. My question is: do I have a way within

pants

to do both of these things together? More specifically, is there a way for pants to run the image as defined as a

docker_image

target in a

BUILD

and then run the tests that aim to use it? (edit: via a single invocation, where pants knows these tests need this container to run) I have tried some various exploration using dependencies, and looked through the docs to see if something similar was outlined there but have come up empty handed. Or is this just not something that I can currently do?

is there a more robust way to get 3rd-party dependencies out of a project in Pants than parsing output of the

./pants dependencies project::

or

./pants peek project::

commands? I was relying on

./pants dependencies project:: --dependencies-type=3rdparty

but this option was deprecated in

2.9.0

. The help docs say that a more robust way of querying dependencies should be available in the later versions, but I struggle to find any.

How can I publish my Docker image using the

registries

field to another container? I.e. I am running pants from a container that is in the same network as my localstack container. I can access it with

localstack

as the host and not

localhost

. But with

registries=[
        "localstack:4510",
    ],

It is trying to reach Get

<https://localstack:4510/v2/>": dial tcp: lookup localstack on 192.168.65.5:53: no such host

Hi! Does

resolve

respect filesystem hierarchy? e.g. if I have something like this:

BUILD(a)
...
foo/
   BUILD(b)
   ...

If I specify in

BUILD(a)

:

python_sources(resolve="my-resolve")

but don’t specify a resolve in

BUILD(b)

, does it / can we make it follow what’s in

BUILD(a)

by default?