An open question on what we want for CLI specs: when you run

./pants test src/py

, do you expect it to only run tests in

src/py

? 🧵

I'm not sure if there are known output flakiness issues with pants / pantsd. I filed https://github.com/pantsbuild/pants/issues/13276 since I couldn't find an existing issue, but not being able to find one surprised me since I feel certain we've had issues like these at least in the past year.

Hi Guys, I want to see the dependency graph of a given target, including third party, how can I do that?

./pants dependencies

doesn’t seem to have this option

Hi Guys I have a couple of questions: 1. Is it possible to use pex_binary with environment variables? 2. We have tests that need an environment variable only if they use a specific library (models django orm - they need the DJANGO_SETTINGS_MODULE), is it possible to pass the variable up the stack to a test? 3. it seems that

pants tailor

ignores fixtures/marks in pytest (

//:pytest-asyncio

if there’s a test that uses

@pytest.mark.asyncio

) , would it be possible detect this automatically 4. we have a

pyproject.toml

that has an

addopts

for example

addopts ="-p no:dash"

, pytest fails because it doesn’t recognize this, we needed to add

--

in the beginning of

addopts

, is this intended or a bug?

Another question, when changing a dependency for a test/pex binary target, we get a relatively big penalty (anything from 60-250 seconds), which rather large and really hurts the feedback loop, is this expected?

Building requirements.pex with 75 requirements: aiobotocore==1.4.2; ...

Takes most of the time

Spotted on CI of https://github.com/pantsbuild/pants/pull/13286/checks?check_run_id=3920580662

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/runner/work/pants/pants/src/python/pants/engine/streaming_workunit_handler.py", line 264, in run
    self.poll_workunits(finished=False)
  File "/home/runner/work/pants/pants/src/python/pants/engine/streaming_workunit_handler.py", line 257, in poll_workunits
    context=self.context,
  File "/home/runner/.cache/pants/named_caches/pex_root/venvs/short/696b29ac/lib/python3.7/site-packages/toolchain/pants/buildsense/reporter.py", line 182, in __call__
    finished=finished,
  File "/home/runner/.cache/pants/named_caches/pex_root/venvs/short/696b29ac/lib/python3.7/site-packages/toolchain/pants/buildsense/reporter.py", line 208, in handle_workunits
    self._build_state.queue_workunits(self._call_count, work_units_map)
  File "/home/runner/.cache/pants/named_caches/pex_root/venvs/short/696b29ac/lib/python3.7/site-packages/toolchain/pants/buildsense/state.py", line 85, in queue_workunits
    self._workunits_chunks_queue.put_nowait((call_num, workunits, timestamp))
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/queue.py", line 190, in put_nowait
    return self.put(item, block=False)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/queue.py", line 136, in put
    raise Full
queue.Full

Any tips for telling

pants.backend.python.goals.run_pex_binary.create_pex_binary_run_request

"no really, the

python

you'd find by running

which python

is a python"? Right now it's only finding a different one

I need help with this rule graph error. The context is I added a rule to infer dependencies for a field that already had a dep inference rule.

re nailgun, what controls the number of nailgun servers that are spawned? context: while working on the Java plugin, I see 16

Starting new nailgun server with cmd

log lines which seems excessive.

@witty-crayon-22786 / @bored-art-40741: I’m triggering the condition at https://github.com/pantsbuild/pants/blob/372b368aa08e667b5e4557082be4f0a7c2a2277e/src/python/pants/backend/java/compile/javac.py#L73-L79 when switching from file sibling deps to same pacage type inference for Java. what does that condition represent?

Does anyone recognise:

ModuleNotFoundError: No module named 'pants.bin'

when trying to build a cross-platform pex of pants? Running

PANTS_PEX_RELEASE=STABLE ./build-support/bin/release.sh build-universal-pex

at 2.8.0.dev5 I'm getting:

[=== 00:44 Checking prebuilt wheels for 2.8.0.dev5+git3ee3288e ===]
All 2 pantsbuild.pants packages were fetched and are valid.
/Users/dwh/.cache/pants/named_caches/pex_root/installed_wheels/a62f822b62a14ee4072eb45dc8505f01ca805f7b/pex-2.1.51-py2.py3-none-any.whl/pex/commands/command.py:412: PEXWarning: The pex root has been set via PEX_ROOT but --disable-cache is also set. Ignoring PEX_ROOT and disabling caches.
  "Ignoring {via} and disabling caches.".format(via=set_via)
Built /Users/dwh/src/github.com/pantsbuild/pants/dist/deploy/pex/pants.2.8.0.dev5.pex
Traceback (most recent call last):
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/pex.py", line 476, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/pex.py", line 532, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/pex.py", line 668, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/pex.py", line 699, in execute_pkg_resources
    runner = entry.resolve()
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/6fc821a7c78f8fce0d8ae58c091013a1d21db5c8/.bootstrap/pex/vendor/_vendored/setuptools/pkg_resources/__init__.py", line 2481, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'pants.bin'
Traceback (most recent call last):
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/b9fbddc60b329b36bd6656142ccba79f43ae4b6e/.bootstrap/pex/pex.py", line 476, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/b9fbddc60b329b36bd6656142ccba79f43ae4b6e/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/b9fbddc60b329b36bd6656142ccba79f43ae4b6e/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/b9fbddc60b329b36bd6656142ccba79f43ae4b6e/.bootstrap/pex/pex.py", line 532, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/b9fbddc60b329b36bd6656142ccba79f43ae4b6e/.bootstrap/pex/pex.py", line 679, in execute_entry
    return self.execute_module(entry_point, alter_sys)
  File "/Users/dwh/.cache/pants/named_caches/pex_root/unzipped_pexes/b9fbddc60b329b36bd6656142ccba79f43ae4b6e/.bootstrap/pex/pex.py", line 691, in execute_module
    runpy.run_module(module_name, run_name="__main__", alter_sys=alter_sys)
  File "/usr/local/Cellar/python@3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/Cellar/python@3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/Cellar/python@3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/dwh/src/github.com/pantsbuild/pants/.pants.d/tmpyxyw4scb/build-support/bin/_release_helper.py", line 1121, in <module>
    main()
  File "/Users/dwh/src/github.com/pantsbuild/pants/.pants.d/tmpyxyw4scb/build-support/bin/_release_helper.py", line 1107, in main
    build_pex(fetch=True)
  File "/Users/dwh/src/github.com/pantsbuild/pants/.pants.d/tmpyxyw4scb/build-support/bin/_release_helper.py", line 732, in build_pex
    subprocess.run([sys.executable, str(dest), "--version"], check=True)
  File "/usr/local/Cellar/python@3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/local/opt/python@3.7/bin/python3.7', '/Users/dwh/src/github.com/pantsbuild/pants/dist/deploy/pex/pants.2.8.0.dev5.pex', '--version']' returned non-zero exit status 1.

@witty-crayon-22786: I am trying to get

test_compile_multiple_source_files

in

src/python/pants/backend/java/compile/javac_test.py

working with same package dependency inference. That test creates two files in the same package but they do not refer to each other’s types, so they should end up in separate `CoarsenedTarget`’s. However, it appears the shared dependency of the

coursier_lockfile

is being found to link them together, and so they end up in a single CoarsenedTarget. Is this expected?

@bored-art-40741: hey! what was the theory on falling back from types to packages here: https://github.com/pantsbuild/pants/blob/feae4861a17ff3ce0d9e85ffe808810bbe2b4786/[…]/pants/backend/java/dependency_inference/package_prefix_tree.py

Hi, I see that running just running pants e.g.

./pants

without arguments takes about 1s

still happening in CI:

Exception: Error linking /home/runner/.cache/pants/named_caches/coursier -> /tmp/process-executionJfc3dg/.cache for local execution: Os { code: 17, kind: AlreadyExists, message: "File exists" }

FYI @witty-crayon-22786 @chilly-magazine-21545 I just saw Issue 10864: Improve MyPy performance which talks about MyPy's persistent cache and challenges because of it. I'm volunteering some personal time to implement something similar for

astroid

issue link, which powers

pylint

. The overall approach isn't dissimilar from `mypy`'s initial implementation (per-module JSON stored in a cache dir. Look at byte-length+mtime to know whether to re-parse). Happy to take future requests to make it

pants

-compatible

I’m getting a lint failure on a

BUILD

file that I can’t replicate on my M1 (pwith py3.9) or on a linux box (with py3.7). There’s no real error in the CI output other than what the file is. How can I diagnose this?

@witty-crayon-22786 given your suggestion about

./pants fix

replacing

tailor

and parts of

update-build-files

...should we add this goal in Pants 2.8 and switch autoflake and pyupgrade to use it, rather than

fmt

? Seems like we need to get that right before stabilizing those new features in Pants 2.8

Rust v1.56.0 is out (with release of Rust 2021 edition). https://blog.rust-lang.org/2021/10/21/Rust-1.56.0.html

hey folks! we’ll likely cut 2.8.0rc0 tomorrow: if you have anything you want to try and land before we cut, lemme know

JVM folks: i’ve updated https://github.com/pantsbuild/pants/projects/22?fullscreen=true to move Scala into the MVP, and to reorder tickets

@hundreds-father-404: looks like

gzip

isn’t as common as

tar

or

unzip

, so i’m thinking of having https://github.com/pantsbuild/pants/pull/13335 use the Python

gzip

module

re: RuleRunner not including environment vars by default:

@hundreds-father-404: thread for discussing the dependency issue with Go plugin and replace directives in transitive dependencies.

@fast-nail-55400: related to the above: do you think you’ll have time to continue with https://github.com/pantsbuild/pants/issues/13152, or is

go

more of a focus for you this week?

I’m getting a failure to find java when running

./pants lint check src/python/pants/backend/java::

on latest

main

(plus my PR for third-party package dep inference):

ValueError: Failed to locate Java for JDK `adopt:1.11`:
setrlimit to increase file descriptor limit failed, errno 22
Downloading <https://github.com/coursier/jvm-index/raw/master/index.json>
Downloaded <https://github.com/coursier/jvm-index/raw/master/index.json>
Downloading <https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11%2B28/OpenJDK11-jdk_x64_mac_hotspot_11_28.tar.gz>
/bin/bash: /bin/java: No such file or directory

I’ve got a CI test failure here that I can’t reproduce on Mac OS (Py3.9) or Linux (Py3.7). Any idea how we normally go about debugging something like this?

FYI once CI passes, I’m going to merge https://github.com/pantsbuild/pants/pull/13295 — this will break JVM resolves until you amend your configuration (ping @fast-nail-55400 @witty-crayon-22786)

i talked with @happy-kitchen-89482 a bit more about https://github.com/pantsbuild/pants/issues/13283, and i think that there would be some definite advantages to actually generating (in memory) transitive 3rdparty dependency `Target`s for more languages (currently just

go

): in particular,

./pants dependencies --transitive

would actually do what users have always asked for, and show transitive thirdparty dependencies as well (with working

peek

)

this is… seemingly quite promising? https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0/