Hello Pants, any timeline for JVM support?

Is there any way standard way in pants to add some additional validation to proto?

proto3

is very relaxed for the sake of compatibility, but am looking for a way not to have to write validation into the endpoints. I'm looking for something that after we deserialize the proto we can check for what we consider a valid message that is a subset of what proto would allow, string not being empty, int's not being 0 etc. Something like a

jsonschema

validator layer for JSON. I was looking at

protoc-gen-validate

but it seems like I would have to swap out the whole

proto

compiler and give it access to some

go

resources. https://github.com/envoyproxy/protoc-gen-validate wondering if anyone else has found a good solution.

\ o / I've nearly monorepod enough libraries to deploy a service. But the "db library" (read: pile of unnecessary garbage gluing sqlalchemy into... nothing?) doesn't do database isolation very well and as such all the tests fail when run in parallel 😂 SO CLOSE.

I have a fundamental multi-repo concept question... in Rust or Java when we make a multi repo it works great because all the dependencies are managed at compile time so we can make sure we don't pull in modules from the wrong place. But python doesn't have this compile time step so things like importing things from the wrong sub-package works when we might not want that. (e.g. I wouldn't want a simple microservice to accidentally pull in

kitchen_sink

). Is it possible to make the following line fail? https://github.com/ehiggs/pants-poetry-test-repos/blob/master/multirepo-1/project-app-async/app.py#L7

https://twitter.com/ToolchainLabs/status/1422604499567452171 Pants Build retweeted: Build Systems Survey http://toolcha.in/survey2021d is a comprehensive study of the developer workflow ecosystem. We hope… https://twitter.com/i/web/status/1422604499567452171 Twitter

I just updated Pants version from 2.5.1 to 2.6.0. When I run

./pants fmt ::

, it tells me that it reformatted one file. When I run

./pants lint ::

it tells me that the file needs to be reformatted 🙂 What could be wrong?

I'm getting failures running Pants test with long branch names in our managed Jenkins. I see this warning:

PEXWarning: The venv for /home/jenkins/agent/workspace/project1_fix_deployment_strategy/.pants/named_caches/pex_root/pip.pex/46820cb5af0dcf9295a4e7f30184cc0e9fa063dc at /home/jenkins/agent/workspace/project1_fix_deployment_strategy/.pants/named_caches/pex_root/venvs/18ed2ab08053f0622ffe35dae7fa5ecfb623151e/206b0fe3aff3bc9983d04b26cf1a4f301f8b293e has script shebangs of '#!/home/jenkins/agent/workspace/project1_fix_deployment_strategy/.pants/named_caches/pex_root/venvs/short/ac7a1b1b/bin/python3.8 -sE' with 132 characters. On some systems this may be too long and cause problems running the venv scripts. You may be able adjust PEX_ROOT from /home/jenkins/agent/workspace/project1_fix_deployment_strategy/.pants/named_caches/pex_root to a shorter path as a work-around.

and this error:

00:03:38.947    File "/home/jenkins/agent/workspace/project1_fix_deployment_strategy/.pants/setup/bootstrap-Linux-x86_64/2.5.1_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 498, in _raise_on_error
00:03:38.947      raise ExecutionError(
00:03:38.947  pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:
00:03:38.947  
00:03:38.948    ProcessExecutionFailure: Process 'Resolving plugins: hdrhistogram' failed with exit code 1.

Apart from choosing a shorter branch name, is there anything that can be done here? I don't have control over the jenkins workspace directory.

FYI, interesting migration strategy for 3rdparty deps I found while helping a friend adopt Pants in their monorepo:

Hi, I’ve tried to install Pants on my MacBook Pro (OSX 10.14.6). First try:

pipx install pants

-> failed, with some hints that this was not the right

pants

package. Second try:

pipx install pantsbuild.pants

-> failed, with the following error:

~/ghq/github.com/pantsbuild/example-python  main  pipx install pantsbuild.pants           ✘ 1  34s  17:44:33
Fatal error from pip prevented installation. Full pip output in file:
    /Users/fermigier/.local/pipx/logs/cmd_2021-08-05_17.49.52_pip_errors.log

pip failed to build package:
    Markdown

Some possibly relevant errors from pip install:
    AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'
    AttributeError: 'HTMLParser' object has no attribute 'unescape'

Error installing pantsbuild-pants.

Third try, after reading the documentation:

~/tmp/pants  curl -L -o ./pants <https://pantsbuild.github.io/setup/pants> && \                      ✔  17:52:14
                   chmod +x ./pants
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   162  100   162    0     0   6812      0 --:--:-- --:--:-- --:--:--  7043
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 13595  100 13595    0     0  67103      0 --:--:-- --:--:-- --:--:-- 67103
 ~/tmp/pants  printf '[GLOBAL]\npants_version = "2.6.0"\n' > pants.toml                             ✔  17:52:16

 ~/tmp/pants  ls                                                                                    ✔  17:52:22
pants      pants.toml
 ~/tmp/pants  ./pants                                                                               ✔  17:52:23
Bootstrapping Pants using /Users/fermigier/.pyenv/shims/python3.9
Installing pantsbuild.pants==2.6.0 into a virtual environment at /Users/fermigier/.cache/pants/setup/bootstrap-Darwin-x86_64/2.6.0_py39
created virtual environment CPython3.9.6.final.0-64 in 561ms
  creator CPython3Posix(dest=/Users/fermigier/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.s3V8wr/install, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/fermigier/Library/Application Support/virtualenv)
    added seed packages: pip==21.1.3, setuptools==57.1.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Requirement already satisfied: pip in /Users/fermigier/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.s3V8wr/install/lib/python3.9/site-packages (21.1.3)
Collecting pip
  Using cached pip-21.2.2-py3-none-any.whl (1.6 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.1.3
    Uninstalling pip-21.1.3:
      Successfully uninstalled pip-21.1.3
Successfully installed pip-21.2.2
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.6.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0)
ERROR: No matching distribution found for pantsbuild.pants==2.6.0

Is there a way to fix this installation problem ? Also, I’d like to know if installing pants globally (using

pip

or

pipx

) will be supported ? (I’d rather type

pants ...

than

./pants ...

).

Hello, I am trying to use Moto to mock out AWS services for my unit tests. When I attempt to run the unit tests with

pants

they fail due to the error below, which appears to be due to missing

setuptools

in the

pants virtualenv

I am using

poetry version 1.2.0a2

to generate a

constraints.txt

which contains

setuptools

as a requirement Is there a reason why pants is not installing setuptools in the virtualenv?

import pkg_resources
E   ModuleNotFoundError: No module named 'pkg_resources'
/home/vagrant/.cache/pants/named_caches/pex_root/venvs/short/041f67f1/lib/python3.8/site-packages/moto/core/models.py:8: ModuleNotFoundError

I know that I can use

--changed-since

to operate on files and targets that have changed relative to a prior commit. Is there a built-in way that I can flip this around to answer the question "did this particular target (and anything it depends on) have any changes relative to a prior commit?" That is, rather than asking "what changed?", I want to ask "Did this specific thing change?" Thanks in advance.

PSA: https://twitter.com/brettsky/status/1423360362754232320

Is there a way to extract the version of

black

pants want's to use? Pants is using one for linting/formatting, but my IDE decided to install another, and now I get to play formatter-ping-pong (the IDE is using a v_env poduced by: https://www.pantsbuild.org/docs/python-third-party-dependencies#tip-set-up-a-virtual-environment-optional , which doesn't collect the tooling deps)

The blog post about Pants' new Poetry support, by @steep-breakfast-98857, is featured in this week's Awesome Python newsletter! https://twitter.com/pythonlibhunt/status/1423384913223458819

https://twitter.com/PythonLibHunt/status/1423384913223458819 Pants Build retweeted: Awesome #Python Weekly #271 is out https://python.libhunt.com/newsletter/271 Featuring @dallasnews @ThePSF @dangoodin001 @pantsbuild Twitter

@hundreds-father-404 Thanks for

skip_tests

in #12510 - that looks fantastic. Then I can skip the problematic chunks of tests that I haven't got working under pytest yet! (as in, they freeze pytest in endless loops due to some eventlet stuff, so relying on pytest to skip those tests isn't an option)

I'm suddenly and consistently getting pantsd killed (probably OOM) in multiple repos, on MacOS, even after rebooting the machine.

This is both when running from source in the pants repo, and running a release in a proprietary repo

setting

--pantsd-max-memory-usage

to some small number doesn't help

And running with

--no-pantsd

gives me

/Users/runner/.cargo/git/checkouts/lmdb-rs-369bfd26153a2575/06bdfbf/lmdb-sys/lmdb/libraries/liblmdb/:3118: Assertion 'pglast <= env->me_pglast' failed in mdb_freelist_save()
Abort trap: 6

Nuking lmdb_store fixes this

But that error is extremely non-informative

Hi folks, I feel this question probably has been ask a million times, but here I go 😂 Is there an existing integration that can read

pants.toml

and update

.vscode/settings.json

? VSCode has been gaslighting me on some minor formatting setting differences; having an way to auto-sync the two settings and use

pants.toml

as the source of truth will probably make VSCode + Pants a better experience.

I have a repo that uses both pandas and a google library. I'd like to build a pex file that will run on both my mac machine and a linux container image. Since pandas has some bindings under the hood, it requires the pre-built wheels, but google has a core dependency that doesn't have any pre-built wheels (https://pypi.org/project/grpc-google-iam-v1/#files). So when I tried to build the pex for multiple platforms, it tells me that it can't find a matching distribution for that grpc package. I was doing this by specifying something like:

pex_binary(
  name="pex_binary"
  platforms=["linux-x86_64....", "macosx..."]
)

Am I missing some sort of config that would let the google dependency build its own wheel, or does it always have to use a wheel when specifying a platform?

Would folks find this interactive visualization a helpful addition to the docs site and/or README? https://octo-repo-visualization.vercel.app/?repo=pantsbuild%2Fpants

Hello! I've a monorepo laid out as such: mondo - root of the monorepo where there is a pyenv dedicated to the 'top' of this repo and pants installed via bootstrap (mondo-3.8.10 via mondo/.python-project) mondo/src/python/project1 - this project as it's own pyenv (project1-3.8.10 via mondo/src/python/project1/.python-project) mondo/tests/python/project1 - this dir uses the same pyenv as src/python/project1 via mondo/tests/python/project1/.python-project with name project1-3.8.10) pyenv mondo-3.8.10 is a minimal pyenv containing only modules to support pants. pyenv project1-3.8.10 contains all of the required modules for coding project1. In my case the AWS Boto3 modules. pyenv activates/deactivates correctly in the terminal when cd'ing into/out of the venv - so, the local dir .python-version is, I'm presuming, accurate for pants purposes. Most every other goal appears to use the project1 pyenv for the stated goal. e.g. ~/mondo:*./pants package src/python/project1:dist* or ./pants typecheck src/python/project1 all complete successfully. However, when I attempt the test goal the tests fail with modules not found. e.g. ~/mondo:*./pants test tests/python/project1:tests* fails. Tried: • *pants.to*ml interpreter_search_paths = ["<PATH>", "<PYENV_LOCAL>", "<PYENV>"] and exporti PATH=/home/x/.pyenv/versions/project1-3.8.10/bin:$PATH • *pants.to*ml interpreter_search_paths = ["*/home/x/.pyenv/versions/project1-3.8.10/bin"*] • Copy tests/python/project1 to src/python/project1/tests and setting BUILD in src/python/project1 python_tests( name="tests", sources = [ "tests/*_test.py", "tests/*_tests.py", "tests/*_test.pyi", "tests/*_tests.pyi", ], ) ~/mondo:*./pants src/python/project1:tests* tests fail with the same module not found errors. I've confirmed the project1-3.8.10 virtualenv has the modules (mondo-3.8.10 does not have them nor should ever need them). Via a terminal, if I cd to ~/mondo/tests/python/project1 the pyenv project1-3.8.10 is activated. Then *pytest _test.py completes successfully for all tests. I've searched Pants documentation, https://github.com/pantsbuild/pants/issues and this Slack account but I've not found anything illuminating. When, running a test goal - is it a correct assumption that the target pyenv should be used for the tests? Are there any flagrant missteps I've done that would prevent an independent projects tests not to execute under the projects venv? Has anyone else had similar issues? Is the next step opening an issue in GitHub? If so, are there any particular artifacts or information to include beyond what I've described? Thank you!

https://twitter.com/pantsbuild/status/1423711588641558530 ❤️🦀❤️🦀❤️ Twitter Excited to see "jump to definition" functionality land in rustdoc! It's something I've wanted since 2016, but alway… https://twitter.com/i/web/status/1423536578010533888 Twitter

Good Friday all! I'm using poetry for module management. I've placed poetry_requirements() in the projects BUILD. From the digging I've done this is to effectively create a python_requirement_library inline from the tool.poetry.dependencies entries. What I'm getting however is incomplete. Given [tool.poetry.dependencies] backoff = ">=1.10.0" boto3 = ">=1.18.11" botocore = ">=1.21.11" Running ~mondo$ ./pants -ldebug --pex-verbosity=4 package src/python/hank_ai_aws:dist Expected Wheel METADATA ... Classifier: Typing :: Typed Requires-Dist: backoff (>=1.10.0") Requires-Dist: boto3 (>=1.18.11) Requires-Dist: botocore (>=1.21.11) Requires-Dist: hank-ai-lib (==0.0.1) this is a first party module Actual Wheel METADATA ... Classifier: Typing :: Typed Requires-Dist: boto3 (>=1.18.11) Requires-Dist: hank-ai-lib (==0.0.1) Gist Debug Output Looking in the output I don't see any references to python_requirement_library nor poetry_requirements. https://gist.github.com/james-crocker/b1ec7fe1f2f837f3a1a336a2e2f6c2b4 Thank you for your time.

Hey all, I've got an interesting issue. I have an executable that depends on stream-chat. As you can see from the PyPi page it's OS independent. If I

./pants run

the executable it runs fine. However, I'm trying to build a cross-platform

.pex

with:

platforms = [
      'linux-x86_64-cp-37-m',
      'macosx-10.14-x86_64-cp-37-m'
   ]

and I get

stderr:
ERROR: Could not find a version that satisfies the requirement stream-chat
ERROR: No matching distribution found for stream-chat
pid 181794 -> /home/oliver/.pyenv/versions/3.7.10/bin/python3.7 /home/oliver/.cache/pants/named_caches/pex_root/pip.pex/cf4106b4c7898b462f7c172dde686a9747103f1a --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root --log /tmp/process-execution11pC41/.tmp/tmpgo8mme15/pip.log download --dest /tmp/process-execution11pC41/.tmp/tmp3aui1xab/macosx_10_14_x86_64-cp-37-cp37m --platform macosx_10_14_x86_64 --implementation cp --python-version 37 --abi cp37m --only-binary :all: --constraint constraints.txt firebase-admin google-cloud-firestore pydantic stream-chat typing_extensions --index-url <https://pypi.org/simple/> --find-links <https://us-central1-python.pkg.dev/infrastructure-314723/python-deps/> --retries 5 --timeout 15 exited with 1 and STDERR:
None

why would it have trouble getting

stream-chat

when it's OS independent? Note that if I remove the

macosx-10.14...

line from

platforms

the

./pants package

succeeds.

Hello, seems that I'm missing something when trying to run tests with coverage, I get the following error:

16:41:55.24 [WARN] Completed: test - data/lib/image/tests/test_image.py failed (exit code 4).
ERROR: usage: pex [options] [file_or_dir] [file_or_dir] [...]
pex: error: unrecognized arguments: --cov-report= --cov-config=pyproject.toml --cov
  inifile: None
  rootdir: /tmp/process-executionrPt8j7

my

pants.toml

config is:

[pytest]
version = "pytest==6.2.4"
args = ["--no-header", "-vv"]
pytest_plugins = [
  "toml",
  "pytest-icdiff",
  "pygments",
]
timeout_default = 60

[test]
use_coverage = true

[coverage-py]
interpreter_constraints = [">=3.7"]
version = "coverage[toml]==5.5"
report = ["raw", "xml", "html", "json", "console"]
config = "pyproject.toml"