pantsbuild #development

Channels

welcome

github-notifications

development

witty-crayon-22786

04/28/2022, 11:46 PM

@happy-kitchen-89482, @hundreds-father-404: i’ve found a blocking issue for

2.11.0

, so i’m getting a patch and another rc out for that tonight. will cut

2.11.0

first thing tomorrow though.

👍 1

average-vr-56795

04/29/2022, 1:45 PM

I'm assuming nothing ^^ affects the weekly today I'm meant to be doing?

bitter-ability-32190

04/29/2022, 1:47 PM

One of my engs brought up that Bazel has a

--rerun N

flag which will re-run tests a certain # of times to help identify flaky tests. I could implement this in Pytest + Pants'

--force

, but it might be useful for other languages. Anyone got thoughts?

✅ 1

witty-crayon-22786

04/29/2022, 7:20 PM

i am … intensely uncomfortable with what i had to do to get a use of

@union

working, heh. combining

@rule_helper

with https://github.com/pantsbuild/pants/issues/12934 just jumped way up in my mind.

👀 1

witty-crayon-22786

04/29/2022, 8:01 PM

@fast-nail-55400: it’s currently the case that the BSP support requires users to configure a JDK for a project on their own, right?

sparse-lifeguard-95737

04/29/2022, 8:37 PM

I’m working on a plugin for

import-linter

, and trying to debug rule graph errors in the integration tests. I basically copy-paste-adjusted the setups from

mypy

and

flake8

to set things up. the errors I’m seeing look like:

Copy code

No source of dependency ImportLinter for @rule(pants.backend.python.lint.import_linter.rules:88:import_linter_lint_partition(ImportLinterPartition, ImportLinterConfigFile, ImportLinterCustomContracts, ImportLinter) -> LintResult, gets=[Get(PythonSourceFiles, PythonSourceFilesRequest), Get(Pex, RequirementsPexRequest), Get(VenvPex, PexRequest), Get(Digest, MergeDigests), Get(FallibleProcessResult, VenvPexProcess)]). All potential sources were eliminated: []

there’s a separate error like ^^^ for every input type to

import_linter_lint_partition

. the stacktrace is coming out of my

pytest.fixture

for setting up a rule runner:

Copy code

@pytest.fixture
def rule_runner() -> RuleRunner:
    return RuleRunner(
        rules=[
            *import_linter_rules(),
            *import_linter_subsystem_rules(),
            *dependency_inference_rules.rules(),
            *config_files.rules(),
            *target_types_rules.rules(),
            QueryRule(LintResults, [ImportLinterRequest]),
        ],
        target_types=[PythonSourcesGeneratorTarget, PythonRequirementTarget, PythonSourceTarget],
    )

any tips to help me debug what’s wrong with the setup? I keep wondering if the empty array in

All potential sources were eliminated: []

is a sign that the rules aren’t getting collected properly

witty-crayon-22786

04/29/2022, 10:34 PM

Categorizing PR#15283... Unable to categorize PR 15283. HTTP error: 403 rate limit exceeded

🥲

🥲 1

witty-crayon-22786

04/29/2022, 11:13 PM

heads up releasers (@happy-kitchen-89482, @hundreds-father-404, @average-vr-56795, @enough-analyst-54434): there have been two changes to the release docs in 2.11.x: 1. you’ll want to auth to the github API if you haven’t already 2. the special cases around

rc0

now apply to a new release type:

(alpha) releases. the strategy and process pages have been updated.

👍 2

witty-crayon-22786

04/29/2022, 11:40 PM

we should maybe rename the

Incremental adoption

page to “Migrating to Pants” or something. i don’t think i had clicked it, because i assumed that it was for larger repositories, rather than any existing repository

👍 1

hundreds-father-404

05/02/2022, 6:19 PM

2.11 release blog draft. feedback welcomed! https://blog.pantsbuild.org/p/eea18101-3977-4a62-a8c5-3104c22c91c8/

👍 1

👀 2

fresh-cat-90827

05/03/2022, 10:50 AM

as per https://www.pantsbuild.org/docs/troubleshooting#how-to-change-your-cache-directory, one can modify the

PANTS_SETUP_CACHE

env var to tell Pants where to store the cache (for the Pants installation in customer’s repo). Is there a way to modify the cache location when running Pants from sources in its own repo (https://www.pantsbuild.org/docs/running-pants-from-sources)? I.e. I checkout the https://github.com/pantsbuild/pants and would like to have cache stored in some other location when running

./pants

. Defining

PANTS_SETUP_CACHE

doesn’t seem to have any effect.

hundreds-father-404

05/03/2022, 8:42 PM

Anyone else having issues with accessing https://www.pantsbuild.org/docs? It's telling me the server is down, but the website isitdown.com is saying it's fine

✅ 2

hundreds-father-404

05/03/2022, 8:47 PM

@enough-analyst-54434 why put

./pants --version foo bar

in our install guide?

enough-analyst-54434

05/03/2022, 9:23 PM

Not sure how that made it in. I was helping Phebe debug the Readme.com save button not working for her. Good to revert.

👍 1

narrow-vegetable-37489

05/03/2022, 10:21 PM

Hmm, I have a pypi dependency that fails to resolve with

python-lockfile-generator=pex

("No matching distribution found") but works fine with

python-lockfile-generator=poetry

. I'm pretty sure it's because my interpreter constraints wants Python 3.9 and the dependency only publishes wheels for 3.6, 3.7 and 3.8 - which I guess results in poetry happily building from source while pex does not? Happy to file a bug if something isn't right there.

happy-kitchen-89482

05/03/2022, 11:40 PM

When adding a new python tool, what do I need to do to get

./build-support/bin/generate_all_lockfiles.sh

to recognize it? I'm using it in a registered rule, I've added it to the AllTools list, it's set up just like every other tool, but I get

Unrecognized resolve name from the option

--generate-lockfiles-resolve`

sparse-lifeguard-95737

05/04/2022, 2:38 PM

👋 I was hopeful that it’d be easy to implement the fix for https://github.com/pantsbuild/pants/issues/15322, but after digging around the code for a bit I might be stumped. we’re setting the

PATH

to the

.shims

dir in the

DockerBinary

setup here - it’s not obvious to me if there’s a way to inject the absolute path of the final work-dir into that value. I thought about hacking in a

$(pwd)

or something, but looks like the values we set in the process env get escaped before being written to

__run.sh

hundreds-father-404

05/04/2022, 3:55 PM

Hm macOS jobs continue to fail today with

Copy code

./pants: line 64: /Users/runner/.cache/pants/pants_dev_deps/Darwin.x86_64.i386.py37.venv/bin/python: No such file or directory
303

narrow-vegetable-37489

05/04/2022, 10:38 PM

What's the best way to go about debugging why Pants is behaving as if both

colors

and

dynamic_ui

are off? It happened to me randomly the other day on 2.11.0rc2 but sorted itself out after a while for unknown reasons, and today it happened to a co-worker on 2.11.0 as well.

hundreds-father-404

05/04/2022, 11:05 PM

I'm making some wording improvements to

[python-infer].unowned_dependency_behavior

. One issue w/ its

warn

mode is it can be extremely verbose -- we give you general guidance how to solve it, but repeat that with every problematic file! Ditto on ambiguous dependencies Any ideas how we can make this less verbose when in warn mode? "Look at this web page to learn more"? "Look at the log above to learn more"?

wide-energy-11069

05/05/2022, 12:34 AM

Hi 👋 I am on this week’s release, although I will be offline this Friday to next Monday, so was wondering if it would be okay to perform the release on Thursday evening or have someone swap with me this week?

wide-energy-11069

05/05/2022, 6:04 AM

Is pants M1 compatible?

Copy code

$ ./pants goals
...
   Compiling process_execution v0.0.1 (/Users/yic/workspace/oss_pants/src/rust/engine/process_execution)
    Finished release [optimized + debuginfo] target(s) in 3m 27s
Traceback (most recent call last):
  File "/Users/yic/workspace/oss_pants/src/python/pants/bin/pants_loader.py", line 19, in <module>
    from pants.bin.pants_runner import PantsRunner
  File "/Users/yic/workspace/oss_pants/src/python/pants/bin/pants_runner.py", line 13, in <module>
    from pants.engine.environment import CompleteEnvironment
  File "/Users/yic/workspace/oss_pants/src/python/pants/engine/environment.py", line 10, in <module>
    from pants.engine.rules import collect_rules, rule
  File "/Users/yic/workspace/oss_pants/src/python/pants/engine/rules.py", line 25, in <module>
    from pants.engine.engine_aware import SideEffecting
  File "/Users/yic/workspace/oss_pants/src/python/pants/engine/engine_aware.py", line 9, in <module>
    from pants.engine.internals import native_engine
ImportError: dlopen(/Users/yic/workspace/oss_pants/src/python/pants/engine/internals/native_engine.so, 0x0002): tried: '/Users/yic/workspace/oss_pants/src/python/pants/engine/internals/native_engine.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/native_engine.so' (no such file)

It’s okay if not, I can switch back to intel.

wide-energy-11069

05/05/2022, 6:56 AM

and neat feature auto categorizing changes, but got rated limited 😅

Copy code

Categorizing PR#15320... Unable to categorize PR 15320. HTTP error: 403 rate limit exceeded

hundreds-father-404

05/05/2022, 2:31 PM

it does! this is in

main

? maybe

rm -rf src/rust/engine

and i think

src/python/pants/engine/internals/native_engine.so

hundreds-father-404

05/05/2022, 2:32 PM

oh and it must use py39, not 37

hundreds-father-404

05/05/2022, 2:32 PM

w/ release script, set

USE_PY39=true

bitter-ability-32190

05/05/2022, 6:14 PM

I finished my PoC for having a processes be batched for effeciency but split for the cache. It...... works! The code bakes in a lot of assumptions/hacks and the biggest issue/question is how to handle the one process result vs many process results (how to split/merge stdout/stderr) But... for a PoC it lays a neat foundation. First run:

./pants --no-pantsd -ldebug --stats-log fmt --only=black ::

(I only retrofitted black for PoC) shows

local_cache_requests_uncached: 1048

(vs 14 on

main

) When I then run

./pants --no-pantsd -ldebug --stats-log fmt --only=black src/python/pants/backend/python/lint/black/rules.py

I don't see any processes being run for

black

(there are uncached requests/processes for getting the ICs). https://github.com/thejcannon/pants/tree/synthcacheproc

🙌 2

👏 2

wide-energy-11069

05/05/2022, 7:55 PM

Copy code

% git clean -ffdx
...
% MODE=debug USE_PY39=true ./pants
...
    Finished dev [unoptimized + debuginfo] target(s) in 55.23s
Traceback (most recent call last):
  File "/Users/yic/workspace/oss_pants/src/python/pants/bin/pants_loader.py", line 19, in <module>
    from pants.bin.pants_runner import PantsRunner
  File "/Users/yic/workspace/oss_pants/src/python/pants/bin/pants_runner.py", line 13, in <module>
    from pants.engine.environment import CompleteEnvironment
  File "/Users/yic/workspace/oss_pants/src/python/pants/engine/environment.py", line 10, in <module>
    from pants.engine.rules import collect_rules, rule
  File "/Users/yic/workspace/oss_pants/src/python/pants/engine/rules.py", line 25, in <module>
    from pants.engine.engine_aware import SideEffecting
  File "/Users/yic/workspace/oss_pants/src/python/pants/engine/engine_aware.py", line 9, in <module>
    from pants.engine.internals import native_engine
ImportError: dlopen(/Users/yic/workspace/oss_pants/src/python/pants/engine/internals/native_engine.so, 0x0002): tried: '/Users/yic/workspace/oss_pants/src/python/pants/engine/internals/native_engine.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/native_engine.so' (no such file)

wide-energy-11069

05/05/2022, 7:56 PM

3.9 is also on path:

Copy code

% python3                         
Python 3.9.12 (v3.9.12:b28265d7e6, Mar 23 2022, 18:22:40) 
[Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

hundreds-father-404

05/05/2022, 7:58 PM

USE_PY39=true

only works with

release.sh

. Normally I think you set

PY=python3.9

. I have an

.envrc

I use

Copy code

export PY=python3.9
export PANTS_PYTHON_INTERPRETER_CONSTRAINTS=[\'==3.9.*\']
export PANTS_BLACK_INTERPRETER_CONSTRAINTS=[\'==3.9.*\']
export PANTS_POETRY_INTERPRETER_CONSTRAINTS=[\'==3.9.*\']