V2 meaning pex v2?

V2 meaning the V2 engine of Pants. The past 3 or so years, we had been slowly rewriting Pants how we wish we had built it from the start. It used to require a lot of work to get caching and parallelism right, and now those things fall out of the design. Same with extensibility: it’s now much easier to write plugins. The past 6 months, it’s come into fruition to the point where Python is now a much better experience than what we were able to do before. Now, we’re putting some last polish onto the V2 Python experience, and then starting to look into supporting other languages

Speaking of which, if you have a few moments, we would love for you to fill out this short survey about how you use Pants: https://t.co/za6xeOxZ6S It helps us to plan the roadmap for Pants, e.g. which language we next rewrite with the V2 engine.

Sure I'll take a look

FYI plugin authors with custom target types: 1.30.0 will require writing light weight bindings for the new Target API. See https://groups.google.com/forum/#!topic/pants-devel/WsRFODRLVZI for more context. As mentioned there, we would be happy to help you do write these bindings!

Is the recommended pattern for writing repo specific plugins still “add path to python_path” and then building a regular python module on that path? For example in

pants-plugins/src/python

I create a module called

awesome_plugin

. This module would need to have

awesome_plugin.register

defined. Is there a pattern that takes advantage of “auto discovery” of plugins that are in the monorepo as opposed to being loaded via pip via an external repository?

yall mention v1 vs v2 quite a bit, but does that refer to any specific release version? or just to two different versions that exist together in the same codebase but refer to different styles and codepaths?

What's the last version of pants that supports py2?

We're getting

No valid Python interpreter found. For this Pants version, Pants requires Python 3.6+.
Could not find . Please ensure  is on your PATH.

with 1.23.0. The pants documentation still says:

By default, pants uses ['CPython>=2.7,<3']

hello, I’m on pants 1.22 (working on upgrade!), pytest 3.6.4, and seeing this coverage error, it’s pretty random in our build, affects different files and does so very inconsistently. I can’t think of anything that differentiates affected files from unaffected ones others.

internal error
Traceback (most recent call last):
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pytest-3.6.4-py2.py3-none-any.whl/_pytest/main.py", line 178, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pytest-3.6.4-py2.py3-none-any.whl/_pytest/main.py", line 215, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pluggy-0.7.1-py2.py3-none-any.whl/pluggy/hooks.py", line 258, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pluggy-0.7.1-py2.py3-none-any.whl/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pluggy-0.7.1-py2.py3-none-any.whl/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pluggy-0.7.1-py2.py3-none-any.whl/pluggy/callers.py", line 196, in _multicall
    gen.send(outcome)
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pytest_cov-2.4.0-py2.py3-none-any.whl/pytest_cov/plugin.py", line 232, in pytest_runtestloop
    self.cov_controller.finish()
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/pytest_cov-2.4.0-py2.py3-none-any.whl/pytest_cov/engine.py", line 149, in finish
    self.cov.stop()
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/coverage-5.1-cp36-cp36m-manylinux1_x86_64.whl/coverage/control.py", line 687, in combine
    combine_parallel_data(self._data, aliases=aliases, data_paths=data_paths, strict=strict)
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/coverage-5.1-cp36-cp36m-manylinux1_x86_64.whl/coverage/data.py", line 117, in combine_parallel_data
    data.update(new_data, aliases=aliases)
  File "/root/app/.pants.d/test/pytest-prep/CPython-3.6.4/6bb24d957dcc4c9000146a942b5ab04ed81cb1f1/.deps/coverage-5.1-cp36-cp36m-manylinux1_x86_64.whl/coverage/sqldata.py", line 661, in update
    path, this_tracer, other_tracer
coverage.misc.CoverageException: Conflicting file tracer name for '/root/app/.pants.d/pyprep/sources/1f8c4b7129eaaea210c186d4118af8f20a186eb6/company/team/product/project/task.py': '' vs '__pants_backend_python_tasks_pytest_prep_coverage_plugin__.ChrootRemappingPlugin'

Is it something you guys have seen before? Happy to submit a github issue.

I'm still getting this sort of thing locally from time to time:

E       ValueError: Could not initialize store for process cache: "Error making env for store at \"/Users/benjyw/.cache/pants/lmdb_store/processes/8\": Too many open files"

I've got

ulimit -n 10000

in my bashrc

the other possibility we were thinking of was that it was going over a shared memory limit since the errno for that is the same as for too many file descriptors

there are instructions on how to set shared memory limits on osx for postgresql on stackoverflow

i have not successfully used this to troubleshoot that error yet, though. it’s not yet clear that shared memory is the issue

Trouble is that when this happens there doesn't appear to be a way to recover short of rebooting

if it was a shared memory limit, there are multiple different limits you can modify with some arcane osx command, which i think would then free up space without rebooting

see the previous thread on this… there are tests that actually end up leaking a bunch of file handles and hitting the limit

so the test is relevant

thanks stu!

Yes, this is pants running pants, I think

So we don't think normal end users are going to see this?

there is more info in the thread, but: no.

afaict, we should fix those tests.

to be clear: raising the file handle limit from the default (256?) is definitely still necessary. but end users should not be hitting anything that is raised beyond 1k-ish.

Stu, we’ve had users hit this in the wild simply running ./pants. This is not a just Pants running Pants issue

@hundreds-father-404: with what file handle limit though? you absolutely will hit it at 256

but see my response above: which test is this? if it’s reproducible on that test, then that’s a place to investigate.

the last time i investigated it, i saw: https://pantsbuild.slack.com/archives/C0D7TNJHL/p1588641352000000?thread_ts=1588640433.498700

if we have reproductions outside of the pantsbuild/pants repo for users who have already increased their ulimit, then let’s absolutely track those down and fix them.