stderr:
Failed to resolve requirements from PEX environment @ /tmp/process-executionRnfOYF/lockfile.pex.
Needed manylinux_2_31_x86_64-cp-37-cp37m compatible dependencies for:
 1: importlib-metadata>=0.12; python_version < "3.8"
    Required by:
      pluggy 0.13.1
      pytest 6.2.4
    But this pex had no 'importlib-metadata' distributions.

from https://github.com/pantsbuild/pants/runs/3089689188?check_suite_focus=true. I had added a PyPi package to the lockfile in that PR. Does the lockfile have to be generated with Python 3.7? (I had run it with Python 3.8.)

@happy-kitchen-89482 CI is broken on main with this:

Exception: String("Cannot strip prefix chroot/dist from root directory (Digest with hash Fingerprint<0e3f69284c886c8b9b330122c22ed47e469e2f9020c9938fdf6d15c24471aeae>) - root directory didn't contain a directory named chroot but did contain directory named: dist")

https://github.com/pantsbuild/pants/runs/3090103212?check_suite_focus=true#step:9:468 Can I revert your change to unblock @wide-energy-11069 from doing the release?

Maybe I missed it.. but it would be nice if pants made a debug logg about not running a

Process

command due to the result already being present in the cache. I spent a good amount of time head scratching why after some refactorings it didn’t run no more… haha.

Can I get workflow approval for my PR https://github.com/pantsbuild/pants/pull/11347 ? (Not sure what the best place to ask is, so asking here)

re: https://github.com/pantsbuild/pants/pull/12369 : the change in log messages to render “Hit local/remote cache for” is only obvious for processes which already had a level of `Info`: the level wasn’t being respected before

ok, the above was resolved. there is one more interesting aspect which i don’t think that https://github.com/pantsbuild/pants/pull/12369 can tackle, but which is probably worth looking at in a non-bugfix followup:

i’ll do a

2.6.x

rc tomorrow: let me know if there is anything else bound for the branch soon

hey folks, not sure if anyone here is actively working on the monopolist idea but we just had a small breakthrough on our side on the package versioning<>git ordering problem. one of my teams built a prototype tool that given a pants target, will emit a wheel with a sha-tagged version with an explicit version (passed by flag at build time) and we’re gaining lots of experience with this internally. one problem that has emerged is that the version ordering there is brittle to human error and needs to be explicitly managed (e.g. the publisher has to increment this manually to get sane ordering). so we started looking at PEP440 “date based release segments”, which appear to work well to keep things ~mostly aligned to the git history (mod sub-second commit collisions). this is the new version generation scheme we settled on for that tool - which is to generate the version from the commit timestamp and then append the sha as a local version identifier):

[omerta pants2wheel (master)]$ git show -s --date="format:%Y.%-m%d.%H%M%S" --format="%cd+%h" HEAD
2021.719.214135+f345b57c9c997

might be useful to get proper

--upgrade

and range pin behavior.

in a plugin, how do I convert from a generated subtarget to the original target from the BUILD file?

What's the status of lockfile generation for Pants?

./pants lock ::

doesn't work due to

testprojects/

afaict (due to the

badreq

req) and running with

./pants lock src:: tests::

completes, but produces surprising results (with only a pex 2.1.42 -> 2.1.43 change, two other requirements, pystache and types-requests - are removed completely from the lockfile). Afaict there is no script or doc or comment in the generated lockfile that give a hint at what command to run to generate the lockfile.

Hey @enough-analyst-54434, do you know how PyPy interpreter constraints are expressed w/ Pex? I realized there's both the PyPy version and the Python reference version, like PyPy 7.3.5 can be PyPy2.7 or PyPy 3.7 is it

--interpreter-constraint='pypy==2.7.*'

or

--interpreter-constraint='pypy==7.3.5'

?

The new JVM tests are flaking pretty regularly, about 5-10% of builds if I had to put a number on it. Okay if I turn them off until @bored-art-40741 or others have a chance to stabilize?

I have a question regarding dataclasses and

frozen_after_init

. In https://www.pantsbuild.org/docs/style-guide#prefer-dataclasses there’s described how to mutate input parameters during init, but no mention of

InitVar

? https://docs.python.org/3/library/dataclasses.html?highlight=initvar#init-only-variables Uhm.. reading up on it again, is it to circumvent the limitation described in there regarding frozen instances.. ?

There is a tiny performance penalty when using `frozen=True`: 

__init__()

 cannot use simple assignment to initialize fields, and must use 

object.__setattr__()

.

Looking at https://pantsbuild.github.io/setup/pants I notice there are a number of redirects. Nothing strange about it, however, I’m a bit concerned that the first hop is for a HTTP location (which in turn simply 301:s to the HTTPS counterpart) but that means we have one insecure request in the middle of what otherwise seems like a secured communication. Not a biggie, but feels unnecessary.

FYI @witty-crayon-22786, new RFC accepted for up safety https://github.com/sunfishcode/rfcs/blob/main/text/0000-io-safety.md

Hello! I’m working on https://github.com/pantsbuild/pants/issues/10620 at the moment, and have a PR along the lines of what @hundreds-father-404 suggested. At the moment there’s a few things that are arch-dependent still, and I want to figure out whether I should consider the current PR as done and open separate issues for the remaining problems, or continue trying to solve things here. I’ll go through the issues in a thread, momentarily:

fwiw: i am very, very annoyed by our overhead in the case of mypy. there is something fishy going on there (beyond https://github.com/pantsbuild/pants/issues/11270), and i’m allowing myself to get distracted enough to triage it

sorry, just slammed CI with cherry-picks.

are there any examples of generating targets dynamically from a plugin?

hey @witty-crayon-22786, I have a quiet weekend and might sprint on getting PyO3 over the threshold as a personal project. Any ideas on where to go incrementally, outside of getting Stdio ported? Almost everything remaining requires the Scheduler

OK, testing out docker support in the example repo, and funny thing is, it uses protobuf’s 😛

It looks like we have benches checked in which haven't compiled for a while... Do we want to fix them (and have them at least compile on CI), or delete them? (Specifically,

fs/store/benches/store.rs

)

Oh, oof...pip-compile does not support the new pip resolver yet, and no one has started working on it. https://github.com/jazzband/pip-tools/issues/1190 That seems like a regression for Pants's UX and makes me think twice about using pip-compile for the lockfile project

In other deficiencies for pip-compile, I can't get it to generate for a foreign platform. This does not error

./pip_compile.pex --pip-args '--platform=fake'

, suggesting the arg gets ignored. (But weirdly,

--pip-args '--not-an-opt'

does fail)

I think we're good for a couple releases today, including 2.6.0? @witty-crayon-22786 I can run those releases if that sounds good.

shaving a half dozen yaks right now, so in case someone else can get to it: it looks like the options reference hasn’t been updated for v2.6 or v2.7 (missing the remote_cache_warnings option)… maybe also v2.5

@witty-crayon-22786 fyi a Pantsd integration test failed on

main

w/ exit code -6 (sigabrt iirc). I don't see a ticket, want me to create one or you're aware of this? https://github.com/pantsbuild/pants/runs/3184651247?check_suite_focus=true#step:11:499

Hey @enough-analyst-54434, thoughts on how much work it would be for Pants to start shading tools like Pytest by using Pex's current functionality? I've been looking into the alternative of mixing in tools like Pytest into user requirements—the Poetry/Pipenv approach—and not liking the coupling.

Hey Pants Team, our organization is adopting pants for many good reasons (lint checks, project introspect, testing, packaging etc..). Right now we’re in the initial phase of lint checks with some custom plugins. Our biggest pain point of adopting pants is getting consistent runs of pants on all developer machines (we use MacOS). While we have documented setup guides using pyenv and venvs it’s not perfect. Developers still run into to errors that are not so obvious how to fix. An example:

16:29:49.96 [INFO] Starting: Building isort.pex with 2 requirements: isort[pyproject]>=5.5.1,<5.6, setuptools
16:30:01.34 [INFO] Completed: Building isort.pex with 2 requirements: isort[pyproject]>=5.5.1,<5.6, setuptools
16:30:01.34 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 489, in _raise_on_error
    raise ExecutionError(
Exception message: 1 Exception encountered:
Engine traceback:
  in select
  in pants.core.goals.fmt.fmt
  in pants.backend.python.lint.python_fmt.format_python_target
  in pants.core.goals.fmt.enrich_fmt_result
  in custom_isort.rules.isort_fmt
  in custom_isort.rules.setup_isort
  in pants.backend.python.util_rules.pex.create_venv_pex (isort.pex)
  in pants.backend.python.util_rules.pex.build_pex (isort.pex)
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/Users/developer/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.1rc3_py38/lib/python3.8/site-packages/pants/engine/process.py", line 253, in fallible_to_exec_result_or_raise
    raise ProcessExecutionFailure(
pants.engine.process.ProcessExecutionFailure: Process 'Building isort.pex with 2 requirements: isort[pyproject]>=5.5.1,<5.6, setuptools' failed with exit code 1.
stdout:

the solution for the above error was to upgrade pip but this is not obvious to most developers. My troubleshooting for pants errors usually goes like this: 1. what’s your python version 2. clear pants cache 3. try upgrading pip 4. try a virtualenv 5. if none above work just run lint checks directly to unblock (not ideal) Developers environments are widely different, most of the time it’s a battle of python versions and path issues. I’m not sure if there are any good solutions for this but I wanted to share some of experiences so far. Any suggestions to improve our consistency across environments would be greatly appreciated.

apropos of nothing: found this thread (while searching for whether tox support lockfiles for its environments): https://discuss.python.org/t/structured-exchangeable-lock-file-format-requirements-txt-2-0/876