Hello all.I have a question regarding

python_test(s)

. The docs state that

runtime_package_dependencies

puts the packaged resources into chroot. If the

runtime_package_dependencies

is a

docker_image

, is there a way to get the image name/tag that was created?

15:27:54.17 [INFO] Initializing scheduler...
15:27:54.43 [INFO] Scheduler initialized.
15:28:38.57 [ERROR] 1 Exception encountered:
  Exception: Snapshot failed: Failed to digest inputs: Throw { val: Failed to watch filesystem for `/builds/path/to/file.py`: Error { kind: MaxFilesWatch, paths: ["/builds/path/to/file.py] }, python_traceback: "Traceback (no traceback):\n  <pants native internals>\nException: Failed to watch filesystem for `/builds/path/to/file.py`: Error { kind: MaxFilesWatch, paths: [\"/builds/path/to/file.py\"] }", engine_traceback: ["digest_file"] }

Has anyone seen this type of error before? I’m running

./pants list ::

on a Gitlab CI/CD job. This is what my

pants.ci.toml

looks like.

[GLOBAL]
pantsd = false
watch_filesystem = false

local_store_dir = ".cache/pants/lmdb_store"
named_caches_dir = ".cache/pants/named_caches"

process_execution_local_parallelism = 8

hi, I have the following structure: • train/ ◦ semantic_segmentation/ ▪︎ BUILD ▪︎ config/ • docker/ ◦ semantic_segmentation/ ▪︎ BUILD The folder config/ has config files distributed among subfolders. I need to have this folder in the working dir of my docker file: I saw in the docks that the .PEX file cannot included files (that we can use

open

), so inside the build file, I defined the docker file:

docker_image(
    name="gpu",
    dependencies = ["docker:base_docker_gpu", "train/semantic_segmentation"],
    instructions = [
        "FROM base_docker_gpu",
        "COPY train.semantic_segmentation/train.pex /bin",
        "COPY train/semantic_segmentation/config /root/",
    ]
)

But the copy instruction fails (

./pants package docker/semantic_segmentation:gpu

):

COPY failed: file not found in build context or excluded by .dockerignore: stat train/semantic_segmentation/config: file does not exist

I tired to add the config folder as a target with the files target:

files(
    name="semantic_segmentation_config",
    sources=["config/*/*", "config/*.yaml", "config/*/*.yaml"],
)

So, what would be the right way to add the config/ folder to my docker image using?

To what extent is the performance impact of caching / not caching

.cache/pants/named_caches

? I noticed that this directory is extremely large (on the order of gigabytes), and caching it is somewhat expensive because it takes a long time to download (and upload) this cache

Hi folks, if you care about macOS 10.15 (Catalina) support, please chime in on this thread in #development: https://pantsbuild.slack.com/archives/C0D7TNJHL/p1659034332441819

Is there a way to have a docker tag default if not satisfied from an env var?

Hello. I have a question as I'm unsure of the right way here. I have a python package with several levels of sub packages (basically a hierarchy of directories, each with their own

__init__.py

). In the end, though, everything is packaged in a single wheel, and the whole hierarchy forms a coherent app.

./pants tailor

creates a `BUILD`file per package/directory. Is that necessary or should I have a single `BUILD`file on top with a target like

python_sources(sources=["**/*.py"])

instead?

I’m having a play with the pex multipl-platform (platforms attribute of pex_binary) support, and mostly works great. One issue I found was that I needed to set the shebang to explicitly, otherwise it would generate one that didn’t match my constraints. Is this a known issue?

pants.toml:

[python]

interpreter_constraints = [“>=3.9.2,<=3.9.13”]

The above generated a shebang of:

#!/usr/bin/env python3.10

If I run pants in the generated venv python-default I get

#!/usr/bin/env python3.9

but I didn’t expect I needed to run pants in the generated venv for it to decide which shebang to use.

Hello everyone, is there a way to install Pants without the

pants

script? Or maybe an already built Pants Docker image (I can't seem to find one)? I ask because we are using Concourse CI and I want to create a Docker image with Pants pre-installed.

Hi everyone 👋 Looking for best practices regarding committing BUILD files. It appears that it’s not necessary (adding

./pants tailor

to CICD), but gitignore recommendations don’t specify BUILD files so I’m afraid I’ve missed something 🙂

I have an issue I'm not sure how to debug. I try to run

./pants test a/test_file.py

on my python/pytest repo and that call pulls a dozen of third-party dependencies, which looks "normal" to me. The problem is that the test doesn't exec. It takes around 80 seconds to build

requirements.pex

and then seems to loop forever when trying to build

pytest_runner.pex

before my machine goes out of memory. I'm not sure what I did wrong or how to debug that 😕

QQ: In Slack people refer to the practice of having one BUILD file per directory as "111". What does the 3rd "1" represent? I assume the first two are "1 BUILD file:1 directory" but I don't know what is implied by the last 1.

In my app, I have a sub-package having sub-packages that looks like the following:

a
├── ba
│   ├── __init__.py
│   └── resources.py
├── bb
│   ├── __init__.py
│   └── resources.py
├── bc
│   ├── __init__.py
│   └── resources.py
├── bd
│   ├── __init__.py
│   └── resources.py
├── be
│   ├── __init__.py
│   └── resources.py
├── bf
│   ├── __init__.py
│   └── resources.py
└── __init__.py

a/__init__.py

uses

importlib.import_module

to load all the

resources.py

files under it (done to do auto-registration on a factory). This isn't picked up by the pants' inference system. I thought to have

a/BUILD

with something like

python_sources(
  overrides={
    "__init__.py": {
        "dependencies": ["*/**/*.py"]
    }
  }
)

But pants is complaining. I'm not sure I can use glob on

dependencies

anyway. Therefore, I'm not sure how to handle that situation. Can you guys tell me if my approach is ok and what is missing to make it works?

Is there a way to abuse parametrized targets to make say, one target for tests and prod that excludes

TYPE_CHECKING

deps, while another goes with the default and uses them?

I think I couldn’t properly setup 3rd party library. Anyone can help me? I am working on django with example-django reference. log_request_id is in requirements.txt, also lockfile (lockfiles > python-defaults.txt)

 ./pants run payhere/app_services/hero_cx/manage.py -- runserver
Traceback (most recent call last):
  File "/Users/szto/.pyenv/versions/3.7.12/lib/python3.7/logging/config.py", line 384, in resolve
    found = self.importer(used)
ModuleNotFoundError: No module named 'log_request_id'

The above exception was the direct cause of the following exception:

I could not see the 3rd party lib from dependencies.

 ./pants dependencies payhere/app_services/hero_cx/manage.py    
payhere/app_services/hero_cx/__init__.py:lib
payhere/app_services/hero_cx/gunicorn.py:lib
payhere/app_services/hero_cx/manage.py:lib
payhere/app_services/hero_cx/settings.py:lib
payhere/app_services/hero_cx/urls.py:lib
payhere/app_services/util/app_service.py
payhere/hero_cx/__init__.py
payhere/hero_cx/apps.py
payhere/hero_cx/swagger.py
payhere/sellers/__init__.py
payhere/sellers/apps.py
payhere/sellers/forms.py
payhere/sellers/signals.py
payhere/sellers/swagger.py
payhere/sellers/tasks.py
payhere/sellers/views.py

Here is my BUILD file

python_sources(
    name="lib",
    dependencies=[
        "payhere/hero_cx",
        "payhere/sellers"
    ]
)

pex_binary(
    name="gunicorn",
    entry_point="gunicorn.py",
    dependencies=[
        ":lib",
    ],
    restartable=True,
)

pex_binary(
    name="manage",
    entry_point="manage.py",
    dependencies=[
        ":lib",
    ],
    restartable=True,
)

I'm trying to do both java and scala codegen from thrift files. I have specified separate namespaces for each language. With the java codegen backend alone,

export-codegen

produces the java: dist/codegen/src/thrift/thrift_example/thriftjava/Person.java but with both java and scala enabled, it produces only the scala: dist/codegen/src/thrift/thrift_example/thriftscala/Person.scala https://github.com/mgedigian-etsy/pants-jvm-codegen/compare/codegen?expand=1

Hello, I was wondering: can I use pants to detect unused files? I'd like to have that check to ensure all dependencies are taken into account and that I don't have dead modules still around. I was imagining something like any python_source target that isn't a dependency of a package/test goal would be reported. Is it possible and is that a good idea?

Hi there 👋 Is there a

pants.toml

schema definition? With that I can have autocomplete on VSCode 🥺

Hi I'm trying to run a Django database management command (specifically,

make_migration

) in a Pants repository. I need to do this outside of the sandbox, since they generate some new files for the codebase. Looking at #12129 and previous discussions, it seems people were working on adding a

run_in_sandbox

flag to

pex_binary

, but it got reverted in #15849. What would be the recommended way to do this now?

Has anyone used Pants with a MicroPython project? I have one starting this week, and I'm just wondering if there are any hiccups I should keep my eyes out for?

root
L config
  L settings
    L test.py
L projects
  L app
    L tests
      L test_app.py

To test test_app.py, I need to add pytest config file to config.settings.test, but test couldn’t get the location. Anyone can help me? BUILD file

python_sources(
    name="lib",
    dependencies=[
        "config",
    ]
)

python_tests(
    name="tests",
    dependencies=[
        ":lib",
    ],
)

Errors

File "/Users/szto/.cache/pants/named_caches/pex_root/venvs/s/a852a762/venv/lib/python3.7/site-packages/pytest_django/plugin.py", line 183, in _handle_import_error
    raise ImportError(msg)
ImportError: No module named 'config.settings.test'

Is there a way to make pants run pytest tests in a batch? The OH of running each in a separate process and creating a pex for it causes about 3 times slow down for a "small" lib by just switching out

pytest <module>

with

./pants test <module>::

in our github actions.

Hi I want to build a python based docker image, but I only want the sources requirements installed. Or more specifically I want the sources dependencies installed in the system python env Is this possible?

I’m curious to hear how different people are doing changelogs in monorepos. I’ve been eyeing https://github.com/changesets/changesets

I just ran command

./pants tailor ::

on a new repository and got:

13:56:24.86 [ERROR] 1 Exception encountered:

  ValueError: The tailor goal only accepts literal directories as arguments, but you specified: ::.  You can also specify no arguments to run against the entire repository.

Running

./pants tailor

works fine. The command with the

::

target is provided in the docs: Generate BUILD files. Is this expected or is it an issue? Perhaps the semantics of the command has changed in a recent version and docs need to be updated?

I recently upgraded from pants 2.11 to 2.12, and I'm getting an error with my proto -> python generation. The generated python code has the following invalid import:

from google.protobuf.internal import builder as _builder
E   ImportError: cannot import name 'builder' from 'google.protobuf.internal'

I'm using

protobuf==3.19.4

in the project. Unfortunately I have a requirement in the project that requires

protobuf<3.20

(due to Tensorflow) so I can't bump this. Anyone have any suggestions? 🙏

Hi. We have introduced Pants in our monorepo project recently and it works fine with python, but we also have projects written in Go. We use go 1.18 with new feature Workspaces, so part of our internal libs are resolved in

workspace.go

, but not in

go.mod

Every lib has own

go.mod

. I have found this https://github.com/pantsbuild/pants/issues/13114#issuecomment-1185321565 comment and not sure, that I’m getting the point. Right now, I cannot build/run the service, because the

lib

is not found. I have tried with

go_third_party_package(
    name="psykhe-common",
    import_path="go/lib/common",
)

but

go_third_party_package

is expected to be generated from

go.mod

file.

Note: for now, Pants only supports repositories using a single

go.mod

. Please comment on #13114 if you need support for greater than one

go.mod

so that we can prioritize adding support.

Does it mean that I cannot use pants for managing GO projects because I have more than 1

go.mod

file in the monorepo?

Hi, I am using this very old pants

1.17.0

. Recently I got dependency issue when building the pex.

...
import attr
File "/xxx/.pants.d/test/pytest-prep/CPython-2.7.12/0ef27f900708c3ec08d3ddb7bff04c4eaa061e2c/.deps/attrs-22.1.0-py2.py3-none-any.whl/attr/__init__.py", line 8, in <module>
    from . import converters, exceptions, filters, setters, validators
File "/xxx/.pants.d/test/pytest-prep/CPython-2.7.12/0ef27f900708c3ec08d3ddb7bff04c4eaa061e2c/.deps/attrs-22.1.0-py2.py3-none-any.whl/attr/converters.py", line 10, in <module>
    from ._compat import _AnnotationExtractor
File "/xxx/.pants.d/test/pytest-prep/CPython-2.7.12/0ef27f900708c3ec08d3ddb7bff04c4eaa061e2c/.deps/attrs-22.1.0-py2.py3-none-any.whl/attr/_compat.py", line 11, in <module>
    from collections.abc import Mapping, Sequence  # noqa
ImportError: No module named abc

I have investigated and find out the error comes from the package attrs , it recently updated and not supports python 2.7 anymore, so I have to peg its version to previous one

21.4.0

I have set the

BUILD

file like this:

python_requirement_library(
    name='attrs',
    requirements=[
        python_requirement(name='attrs', requirement='attrs==21.4.0'),
    ],
    dependencies=[
        ':coverage',
    ],
)

But when I run test or build pex, I still got the same error, and when I look into the

.pants.d/python-setup/resolved_requirements/CPython-2.7.12/

, there is still

attrs-22.1.0-py2.py3-none-any.whl

installed, not the

21.4.0

What should I do ? Can anyone help me~? Thanks a lot!

Hello, I'm trying to create a target for a custom codegen. I was wondering where are pants.engine.target and pants.backends. I tried

pip install pants

but it doesn't seem to be in that package.

Hello, I’m trying to determine which Python installation

pants

is using when I call

./pants test ::

for example. When I run the test command with

-ldebug

it appears (I could be wrong) that

pants

is spawning multiple processes with different versions of Python that I have installed--even if I remove them from my

PATH

. For example:

16:42:45.42 [DEBUG] spawned local process as Some(23549) for Process { argv: ["/Users/username/miniconda3_py3.8/envs/io-env/bin/python"...
...
16:42:45.43 [DEBUG] spawned local process as Some(23550) for Process { argv: ["/Users/username/.pyenv/versions/3.8.13/bin/python"...
...
16:42:45.43 [DEBUG] spawned local process as Some(23551) for Process { argv: ["/Users/username/.pyenv/versions/3.10.4/bin/python"...
...

What is going on there?