Hey 👋 One thing I've come up against today when investigating 👖 is having to do:

sources=["**/*.py"]

in a

python_sources

target, so that all

*.py

files in subdirectories are picked up. I'm wondering if I'm doing something wrong 🙂 More context in 🧵

I'm trying to run pants 2.14 within a docker image on a arm mac host, to simulate CI (i.e. where pants is running on Linux), and hit the following error:

$ docker run -it --platform linux/amd64 python:3.9 bash -c 'echo -e "[GLOBAL]\npants_version = \"2.14.0\"" > pants.toml && curl -L -O <https://static.pantsbuild.org/setup/pants> && chmod +x ./pants && ./pants version'
... setup ...
New virtual environment successfully created at /root/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39.
20:52:23.81 [INFO] waiting for pantsd to start...
20:52:28.81 [INFO] waiting for pantsd to start...
Traceback (most recent call last):
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 215, in __init__
    self._py_scheduler = native_engine.scheduler_create(
ValueError: Failed to begin watching the filesystem: Function not implemented (os error 38)
20:52:33.83 [INFO] waiting for pantsd to start...
... continues like that ~forever ...

Is there a way to get this to work? _update_: solution: run like

./pants --no-pantsd --no-watch-filesystem ...

and make sure the docker containers have a high enough memory limit

Just found that Github Action's

ubuntu-latest

got updated and now the default base image only provides Python 3.10.6... I think it's time to make Python 3.10 wheels for Pants! 😉

Since upgrading to 2.14 we're now getting a lot of warnings about inferring owners of imports... Predominantly they're

pytest

. Is this a known thing?

Is there a way to let Pants treat pytest's exit code 5 as a successful run?

./pants export ::

only supports exporting all resolves, I suggest that

./pants freeze --resolve=xyz

should be supported. As a alternative, we can implement

freeze

in the following way: https://github.com/da-tubi/pants-minimal-freeze

Quick question as I couldn't see in the docs or here -- is it possible to have docker images dependent on other docker images, so that if files for a base image change, then the base image is rebuild and downstream images?

Here's a question I never thought about too much - does Pex work on Windows?

I have a python monorepo with a layout like this:

.
└── libs
    ├── a
    │   ├── BUILD
    │   ├── kaiko
    │   │   └── a <- *.py files in there
    │   └── requirements.txt
    ├── b
    │   ├── BUILD
    │   └── kaiko
    │       └── b <- *.py files in there
    └── c
        ├── BUILD
        └── kaiko
            └── c <- *.py files in there

So in the library

a

- that depends on both

b

and

c

- we have imports of the from

from kaiko import b

and

from kaiko import c

, because

kaiko

is a shared namespace. Both

libs/a

,

libs/b

, and

libs/c

are roots. `a`'s

requirements.txt

contains editable installs stanzas (

-e ../b

and

-e ../c

). That doesn't work when typechecking with

mypy

(and

pyright

), I get

Cannot find implementation or library stub for module named "kaiko"

and

Cannot find implementation or library stub for module named "kaiko.b"

errors 😞💥 🏴‍☠️ (depending on the import's shape). Should I use modules and module mapping statements in my

BUILD

file to circumvent this? I clearly see that my setup seems not to fit w.r.t. how pants treats imports, but I don't understand how I would write the mappings.

Oh here's a fun one. An optional dependency by a 3rdparty module on a module you aren't directly consuming 😵 (so it's in the lockfile, but you can't refer it in any BUILD files)

I'm struggling with something that seems like it should be a common objective. I have a monorepo where shared libraries are in a

libraries

directory. I'd like to be able to reference these relatively so that apps and libraries are using the latest source code rather than being referenced via a version-pinned requirement reference. In poetry, I can use path references in pyproject.toml.

[tool.poetry.dependencies]
quoteoftheday = {path = "../quoteoftheday"}

I'd like to configure

path

BUILD files to reflect this. I find my Google searches pointing me to

python_library

in the v1 documentation quite often. Can someone point me toward a solid resource to understand 1. How to properly configure a library distribution 2. How to reference that as a dependency from another project in the same monorepo

is there a reason pants doesn’t accept defining the flake8 version inside your python requirements as opposed to the pants.toml file? Defining inside our

pyproject.toml

is easier for us so we can leverage tools like dependabot/renovate to automatically bump the version

so I tried to upgrade and failed. flake8 6.0.0 supports python 3.8 and above (https://github.com/PyCQA/flake8/blob/b9a7794c4f425ef8419081e6993f99419cc667ea/setup.cfg#L41) pants tries to use it under 3.7 (when calling generate-lockfiles) and fails (this is expected). But unlike other tools (like black) flake 8 doesn't have an

interpreter_constraints

options .... any advice ?

Hi guys I have a question, we stayed in pants 1.30.4 and recently some of our member changed to a M1 processor Mac and is having some issue during setup, is there any way to let pants1 running with M1 processor or upgrade to pants 2 is the only solution? https://www.pantsbuild.org/v1.30/docs/troubleshooting#running-pants-with-m1-macs--apple-silicon I have found this doc, we will try this first.

HI, first of thanks @refined-addition-53644 and @happy-kitchen-89482 for accommodating me the last few days with my questions. want to understand something • in my project I have multiple name space folders like

├── ROOT Repo
├── .gitignore
├── README.md
├── lambda-one
│   └── lambda_one
|        ├── config -> directory containing other python files
|        └── lambda.py -> has import statement pointing to shared-code from shared_code.module_b.file_name import hello_world 
│   ├── tests -> contains test for this 
│   ├── poetry.lock
│   └── pyproject.toml
├── lambda-two
│   └── lambda_two
|        ├── config -> directory containing other python files
|        └── lambda.py -> has import statement pointing to shared-code -> from shared_code.module_b.file_name import sum 
│   ├── tests -> contains test for this 
│   ├── poetry.lock
│   └── pyproject.toml
├── shared-one
│   └── shared_code
|        ├── module_a -> directory containing python files
|        └── module_b -> directory containing python files
│   ├── tests -> contains test for this 
│   ├── poetry.lock
│   └── pyproject.toml
├── pants
└── pants.toml

I understand that pants supports creating of lock files. is it possible for pants to resolve the lock files for each module • I know I can define multiple resolve names and point to something but the problem is those lock files contain the same list of dependencies • what am wondering is if i can resolve the poetry.lock file in the pants.toml from each package and then within my

BUILD

do something like this

# for lambda-one/BUILD
__defaults__(all=dict(resolve="lambda-one"))

poetry_requirements(
    name="poetry",
)

# for lambda-two/BUILD
__defaults__(all=dict(resolve="lambda-two"))

poetry_requirements(
    name="poetry",
)

# for shared-code/BUILD
__defaults__(all=dict(resolve="shared-code"))

poetry_requirements(
    name="poetry",
)

My thinking here is to have pants sort of "lock" each modules dependencies separately. wonder if that makes sense or something worth doing

Two questions on the

docker_image

target: 1. How can I target a Dockerfile with the

source

fild that is not relative to the BUILD file or any subdirectoy but is one level up, i.e.

../Dockerfile

? 2. I have seen that in a BUILD file I can reference values from the

pants.toml

like

repository="{build_argsREPO_URL}"

. Is it possible to get the name of the directory where the BUILD file is located in, such that I can do something like:

extra_build_args=["{THIS_DIRECTORY_NAME}"]

?

Hi all, we’re trying to use the option

[python].resolves_to_no_binary

as explained here. we put in our pants.toml the following:

resolves_to_no_binary = {__default__=["cairocffi"]}

which should do the same as the

--no-binary

option of pip, that let us to install this dependency without errors. Unfortunately we are not able to replicate the behaviour with pants, do you have any idea why?

Any thoughts on best practices for creating unique docker tag values for use with immutable repositories? After some research (this article presents some key considerations), I'm thinking of running with something like •

<trimmed branch name>--<git short commit sha>--<github actions build attempt number>

for CI builds and •

<trimmed branch name>-<git short commit sha>--<timestamp>

for local builds Curious what others are doing, and if an opinionated version (if a starting point) could be added to the docs or a blog post.

I’ve apparently not used the pants introspection features much.. is there no easy way to get the answer to: “give me all

pex_binary

targets that depends on this

python_requirement

” ? I tried

./pants dependees

(still on 2.14) but that gives me too much, and using

--filter

seems to filter my input args, not output results 😬

Hey all, I'm having some grief with

export

and

Pycharm

. After some digging I found that the exported virtualenv has a circular dependency in symlinks 🤯

├── python -> /path/to/repo/dist/export/python/virtualenvs/python-default/3.9.15/bin/python3
├── python3 -> python
├── python3.9 -> python

Very tiny piece of feedback, it would be ideal if I didn't have to override the

name

property of the

python_tests

target, since otherwise it duplicates the

python_sources

target name.

python_sources()
python_tests(name="tests")

hi in poetry project that has its own

pytest

and other blocks defined that pants supports under the backend configuration. will one need remove/duplicate the definitions from the

pyproject.toml

into

pants.toml

or is pants "spart" enough to detect that and use the definitions from

pyproject.toml

. Thanks in advance

hi another poetry related question. normally if i want to execute something from poetry i would be able to run something like

poetry run <dependency

with pants being able to detect my 3rd party dependencies. is there a need to either run

poetry install

to prep my local dev before running my

poetry

commands or does pants support sort of running something from

./pants run poetry

This feels odd, seems like i've named modules "types" before, and it's worked:

12:25:09.08 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Determine Python dependencies for avnpkg/avnpkg/types.py' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/tmp/pants-sandbox-kPx7dX/./__parse_python_dependencies.py", line 11, in <module>
    import ast
  File "/usr/lib64/python3.9/ast.py", line 29, in <module>
    from contextlib import contextmanager, nullcontext
  File "/usr/lib64/python3.9/contextlib.py", line 6, in <module>
    from functools import wraps
  File "/usr/lib64/python3.9/functools.py", line 22, in <module>
    from types import GenericAlias
  File "/tmp/pants-sandbox-kPx7dX/types.py", line 2, in <module>
    from dataclasses import dataclass
  File "/usr/lib64/python3.9/dataclasses.py", line 1, in <module>
    import re
  File "/usr/lib64/python3.9/re.py", line 124, in <module>
    import enum
  File "/usr/lib64/python3.9/enum.py", line 2, in <module>
    from types import MappingProxyType, DynamicClassAttribute
ImportError: cannot import name 'MappingProxyType' from partially initialized module 'types' (most likely due to a circular import) (/tmp/pants-sandbox-kPx7dX/types.py)

is there a way to see what files are being changed when Pants says

13:22:13.05 [INFO] Filesystem changed during run: retrying `Test` in 500ms...

I've tried

-ldebug

but i'm not seeing the information I want. I just see a bunch of things like

13:20:47.09 [DEBUG] Dependency @rule(pants.engine.internals.build_files.evaluate_preludes()) of Some("@rule(pants.engine.internals.build_files.parse_address_family(tests/components/security))") changed.

but that target demonstrably didn't change

I am trying to configure Pants for the Windows Subsystem for Linux. After doing a fresh Ubuntu install, discover that the default Python that has been automatically installed is Python 3.10.6. When I attempt to run Pants with this, I get:

No valid Python interpreter found. For `pants_version = "2.16.0.dev0"`, Pants requires Python 3.7, 3.8, or 3.9 to run. Please check that a valid interpreter is installed and on your $PATH.

From the docs, it appears that Pants will support builds with more recent versions of Python, but Pants itself cannot run with a version of Python newer than 3.9. So I install 3.9 following these instructions: https://www.debugpoint.com/install-python-3-11-ubuntu/ (substituting 3.9 instead of 3.11). I make 3.9 the default following those instructions:

bruce@groot:/mnt/c/git/semaphore-demo-python-pants$ sudo update-alternatives --config python3
There are 2 choices for the alternative python3 (providing /usr/bin/python3).

  Selection    Path                 Priority   Status
------------------------------------------------------------
  0            /usr/bin/python3.10   2         auto mode
  1            /usr/bin/python3.10   2         manual mode
* 2            /usr/bin/python3.9    1         manual mode

But when I try to run Pants I get:

bruce@groot:/mnt/c/git/semaphore-demo-python-pants$ pants test ::
Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 28, in <module>
    from CommandNotFound import CommandNotFound
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 19, in <module>
    from CommandNotFound.db.db import SqliteDatabase
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 5, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

I try running the extra install command from here https://www.pantsbuild.org/docs/prerequisites#microsoft-windows:

bruce@groot:/mnt/c/git/semaphore-demo-python-pants$ sudo apt install unzip python3-dev python3-distutils python3-venv gcc
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
gcc is already the newest version (4:11.2.0-1ubuntu1).
python3-dev is already the newest version (3.10.6-1~22.04).
python3-distutils is already the newest version (3.10.6-1~22.04).
unzip is already the newest version (6.0-26ubuntu3.1).
python3-venv is already the newest version (3.10.6-1~22.04).
0 upgraded, 0 newly installed, 0 to remove and 20 not upgraded.

It seems like I missed a step or two somewhere...

Also it does appear that 3.9 is the default:

bruce@groot:/mnt/c/git/semaphore-demo-python-pants$ python3 --version
Python 3.9.15

Hi all , Trying to push to the docker registry

GIT_COMMIT="jokes_on_you" ./pants publish ::

Am getting the following error

The push refers to repository [<http://docker.io/ineeddockeragain/play|docker.io/ineeddockeragain/play>]
dfc49893f785: Preparing 
07d2d7aa362e: Preparing 
7c1610cf2397: Preparing 
16427a3bb904: Preparing 
449a28044116: Preparing 
f11bbd657c82: Waiting 
denied: requested access to the resource is denied

✕ ineeddockeragain/play:jokes_on_you failed.

Hi all 👋 I am unable to get pandas==1.3.4 installed. getting error:

00:23:21.00 [INFO] Completed: Generate lockfile for python-default
00:23:21.00 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:

stderr:
pid 26138 -> /Users/avishwakarma/.cache/pants/named_caches/pex_root/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/07444ef590b1dd2e7c6803e2e9bf769aed6ea919/bin/python -sE /Users/avishwakarma/.cache/pants/named_caches/pex_root/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/07444ef590b1dd2e7c6803e2e9bf769aed6ea919/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/avishwakarma/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/0t/dmh8ynt13pbc2y2stvb0by6c0000gn/T/pants-sandbox-wm6jpA/.tmp/pex-pip-log.f99pdiq3/pip.log download --dest /private/var/folders/0t/dmh8ynt13pbc2y2stvb0by6c0000gn/T/pants-sandbox-wm6jpA/.tmp/tmpmvf3vkp5/Users.avishwakarma..pyenv.versions.3.7.10.bin.python3.7 pandas==1.3.4 requests==2.26.0 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement pandas==1.3.4
ERROR: No matching distribution found for pandas==1.3.4

Is there a kubernetes/knative goal. I understand there's helm target. But helm seems too bloated for me right now.