Can someone ELI5

export

vs

export-codegen

?

export

- Export Pants data for use in other tools, such as IDEs.

export-codegen

- Write generated files to

dist/codegen

for use outside of Pants. I'm missing some of the nuances as to what determines whether a new plugin should use one over the other, since they both appear to end up in the

dist

folder. https://www.pantsbuild.org/docs/setting-up-an-ide has some additional examples, which are

virtualenv

vs

protobuf

. The best heuristic I can come up with is that

export

is intended for tooling/support/not-part-of-release, while

export-codegen

would represent files which would be part of the compilation/packaging/binary/production release process. Specifically, I would wonder where the bash completions would go, and based on my assumption above, it's more likely an

export

since it's sourced by the terminal, and has no part in actual production code.

Hi, Is there any plan to add pydocstyle as a linter? I was thinking of contributing.

There is the

--no-process-cleanup

flag, that preserves the sandbox directories (and logs their paths) so you can inspect them

And there is the

export-codegen

goal to get generated code out of the sandbox and into dist, as mentioned

I'm not too familiar with Please's

--shell

but it sounds like

--no-process-cleanup

is ~similar

The Scylla and Charybdis of python packaging right here

Hi there. This is maybe a real basic, foundational question, but I'm messing around with pants, and when I ran

./pants tailor

I noticed it recursed down into my python package to add a

BUILD

file to every level of the Python package. Is this required to have all of those

BUILD

files, or can you have one for each python package?

RFC on a blog post highlighting cranking Python+Docker to 11: https://docs.google.com/document/d/1CpyA57MRtOMJHSjuRLoK8_CBJWWBxvPOK1G0j2_zQWA/edit?usp=sharing

Hi folks, I could use some help figuring out how to debug an issue. My team is using pants, and some of our newer employees have had trouble running the coverage tool. We’re using pants 2.10.0 on python 3.9, which everyone is installing via pyenv. We are all on the same M1 macbook pros at the same OS version. We’re using the default lockfile for coverage and pytest. The relevant section of the pants.toml is:

[test]
use_coverage = true

[pytest]
args = ["--no-header", "-vv", "--log-cli-level", "INFO"]

[coverage-py]
report = ["html", "xml", "console"]
config="tooling_configs/.coveragerc"

When they run

./pants test

on a python tests target, pytest runs successfully, and then crashes trying to build the coverage pex:

11:08:42.69 [INFO] Completed: Run Pytest - src/stages/qc_metrics/tests/test_qc_math.py:../tests succeeded.
11:08:42.70 [INFO] Completed: Run Pytest - src/stages/qc_metrics/tests/test_qc_metrics.py:../tests succeeded.

✓ src/stages/qc_metrics/tests/test_qc_math.py:../tests succeeded in 4.91s (memoized).
✓ src/stages/qc_metrics/tests/test_qc_metrics.py:../tests succeeded in 18.90s (memoized).

Wrote test XML to `dist/test_results`
11:08:46.73 [INFO] Completed: Building coverage.pex from coverage-py_default_lockfile.txt
11:08:46.73 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building coverage.pex from coverage-py_default_lockfile.txt' failed with exit code 1.
stdout:
(see thread for more)

It’s very odd that it’s looking for cp38, since we are on 3.9. 3.8 is the system default. And, I don’t have this problem personally, it’s only shown up for two people, but it’s consistently reproducible for them. They’ve already tried clearing the ~/.cache/pants directory to no effect. Does anyone know anything about this, or have an idea how we could go about debugging it further? Any help appreciated.

What's the right way to provide C libraries to a pants project, like if one of my python deps links against libcurl or something

@nice-florist-55958: followup re: multiple resolves:

how is the difference of semantics of “built-in” commands like

build

,

test

,

fmt

,

run

, and

lint

encoded in pants? are the same knobs available for any targets?

Hey folks! My dependency generation is hanging when I try and use a nightly release of PyTorch. I have a downloaded whl inside a local directory. Things work fine with 1.12.0 which is the latest stable release. But when I try and install the nightly (

1.13.0.dev20220723+cu116

) my

generate-lockfiles

just hangs, it’s been going 1700s now. Is this some kind of parsing thing?

I've updated a manual about setting up VSCode/Vim/NeoVim to work with a pants-based Python project: https://docs.backend.ai/en/latest/dev/daily-workflows.html#vim-neovim

Now it includes explicit configuration examples for ALE and CoC (popular Vim plugins for code linting and navigation) as well.

Also I've noticed that https://github.com/lablup/backend.ai/blob/6636e3db145a9778cfed60e7b0ccd9578f0ff69b/pyproject.toml#L56 this line (

python_executable

, in addition to

mypy_path

) is very important to make mypy working with type stubs and external dependencies.

It would be nice to have a fixed path for

dist/export/python/virtualenvs/python-default/3.10.5/bin/python

when changing the Python version, to avoid things like: https://github.com/lablup/backend.ai/blob/6636e3db145a9778cfed60e7b0ccd9578f0ff69b/pants.toml#L40 and letting people update their IDE/editor configurations. Probably having a symbolic link good enough?

Can I specify in-repo pytest plugins somehow for the pytest sub-system? I have in

setup.cfg

[tool:pytest]
...
addopts =
    -p amra.pytest.plugin.suites

When I run

./pants test

, the

amra.pytest.plugin.suites

module is not found, because it is not imported in the tests.

Hello I'm testing pants with my repo containing some Python code and I have troubles to figure out what to put as source roots and where to place BUILD files. ./pants tailor doesn't find anything. My projects have a dir structure with a src and tests directory and I didn't find any examples on the doc with that. It looks like that:

├── pants
├── pants.toml
└── py
    ├── libraries
    │   └── libtutu
    │       ├── LICENSE
    │       ├── pyproject.toml
    │       ├── README.md
    │       ├── setup.cfg
    │       └── src
    │           └── libtutu
    │               └── __init__.py
    └── projects
        ├── tata
        │   ├── LICENSE
        │   ├── pyproject.toml
        │   ├── README.md
        │   ├── setup.cfg
        │   └── src
        │       ├── tata
        │           └── __init__.py
        └── standalone
            ├── LICENSE
            ├── pyproject.toml
            ├── README.md
            ├── setup.cfg
            ├── src
            │   ├── standalone
            │       └── __init__.py
            └── tests
                └── test_dummy.py

Does anyone have an example I could follow for when you have the src/ and tests/ directory?

I have another question. I can't access pypi or any other public repo. Instead, we have artifactory which acts as a "repo proxy" + hosts private repos. I added a [python-repos] sections and a indexes entry with a list of URLs in pants.toml but I'm not sure pex honors it. When trying to run pytest, I get the following messages:

pex.result.ResultError: There were 10 errors downloading required artifacts:
1. pytest-cov 3 from <https://files.pythonhosted.org/packages/20/49/b3e0edec68d81846f519c602ac38af9db86e1e71275528b3e814ae236063/pytest_cov-3.0.0-py3-none-any.whl>
    <urlopen error [Errno 104] Connection reset by peer>
2. pytest 7.0.1 from <https://files.pythonhosted.org/packages/38/93/c7c0bd1e932b287fb948eb9ce5a3d6307c9fc619db1e199f8c8bc5dad95f/pytest-7.0.1-py3-none-any.whl>
    <urlopen error [Errno 104] Connection reset by peer>
3. coverage 6.3.2 from <https://files.pythonhosted.org/packages/60/a8/0f951285dff76cbedbbba89c1969586020150a3b53267b5d8674b2ead15a/coverage-6.3.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl>
    <urlopen error [Errno 104] Connection reset by peer>

Is there another option to set?

Could somebody point me the guide for migration pants plugins from 2.12.0 to 2.13.0rc0? I'm now trying to upgrade pants, and seeing errors related to rule graphs.

Can you remove the Django dependency for some folders in a Pants & Django project? For example, for testing a

shared/utils

folder. Unfortunately the django example github does not seem to have such an example. When I tried this naively, it seems pytest always want to import some kind of Django settings for the test.

Hi everyone! Maybe you can help to understand some Pants machinery. I have enabled code coverage for the project and found that I start receiving such error on CI for each job, where all test results were memoized:

✓ <test_name1>.py succeeded in 10.65s (memoized).
✓ <test_name2>.py succeeded in 19.10s (memoized).
  
  Wrote test reports to .test_results
  Error: 4.58 [ERROR] 1 Exception encountered:

ProcessExecutionFailure: Process 'Generate Pytest report coverage report.' failed with exit code 1.
  stdout:
  No data to report.

Looks like we can't gather coverage data without actual test run? Is my observation is correct? Should I create an issue on the GutHub or am I doing something wrong here?

Hi everyone, what’s the way to define dependency on a distribution from local repo which uses a different resolve. I tried to use the associated target directly but it complains

All dependencies must work with the same

resolve

.

Could

pex_binary

be generalized to

script_binary

for dynamic scripting languages Pants doesn't support yet? I'm imagining a target with similar semantics but with just a script file entry point to be executed by the OS (and so with whatever is in the file's shebang line, i.e., the language's interpreter). Dependencies would just be other first-party source-like targets such as

resources, python_sources, shell_sources, etc.

bundled as a collection of files relative to the entry script (or relocatable to it). A new target

script_sources

instead of

resources

as a generic stand-in could be added too. This would allow other projects based

q/kdb

,

bash

,

nodejs

, or any other interpreted language to formally use Pants, both to enumerate their sources and to package themselves into a single executable file. Migration would also be easier once any given language has first-class support. At present, could the existing

archive

(for the executable) and

resources

(for the sources) targets be post-processed after the Pants packaging step to be executable using a given interpreter path? At the moment, all I can think of is to lightly use Python as a glue language and package a

pex_binary

with an entry point script that just sub-processes out to the actual script that the author wants executed.

I'm interested in trying thrift generation in java and scala. Is it in a state where I can play with it? I don't see it in docs, but I see some implementation in code. I can't read the code well enough to tell how to try using it.

Hey @witty-crayon-22786 / @bitter-ability-32190, I saw that https://github.com/pantsbuild/pants/issues/15026 had the good first issue tag. Since this is a feature that I’m interested in, I was wondering if either of you had any suggestions on how I could get started with investigating how to do this

Why don't we dump the relevant ENV vars in a sandbox's

__run.sh

?

What's the policy on usage when a company gets added to the users page? Is it just the website? Or does pants get to use the logo elsewhere?

Hi All. I need help with an issue I have. I am building a pex with no dependencies. My environment is supposed to have them. I am testing this locally by using pyenv, when I build my image the image is build without dependencies. it seems like the pex setups its environment so that it runs in an isolated environment. I have dependecy errors but the interpreter I am using does contain the dependencies. Is there a way to correctly do this?