https://pantsbuild.org/ logo
Powered by
#github-notifications
  • c

    cool-easter-32542

    05/12/2023, 3:15 PM
    #18994 Support for `docker run` arguments in `docker_environment` Issue created by generalov Is your feature request related to a problem? Please describe. I'd like to specify additional 
    docker run
    arguments for a 
    docker_environment
    . Describe the solution you'd like
    Copy code
    docker_environment(
  ...,
  docker_run_args = ["--add-host <http://pypi.myorg.com:10.0.1.11%22|pypi.myorg.com:10.0.1.11">]
)
    Describe alternatives you've considered Alternatively, 
    docker_environment
    could inherit the 
    docker run
    arguments from 
    [docker].run_args
    in 
    pants.toml
    . Additional context Probably, there should be another tool for spawning complex environments like 
    docker compose
    or devcontainers. pantsbuild/pants
  • c

    cool-easter-32542

    05/13/2023, 12:45 AM
    #2150 A locked requirement with mixed artifact types fails to lock. Issue created by jsirois An example error (with PEX_VERBOSE set) looks like so:
    Copy code
    File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/result.py", line 105, in catch
    return func(*args, **kwargs)
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/cli/command.py", line 84, in run
    return subcommand_func(self)
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/cli/commands/lock.py", line 482, in _create
    create(
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/resolve/lockfile/create.py", line 428, in create
    locked_resolves = lock_observer.lock(downloaded)
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/resolve/lockfile/create.py", line 328, in lock
    return tuple(
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/resolve/lockfile/create.py", line 329, in <genexpr>
    LockedResolve.create(
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/resolve/locked_resolve.py", line 534, in create
    return cls(locked_requirements=SortedTuple(locked_requirements), platform_tag=platform_tag)
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/sorted_tuple.py", line 60, in __new__
    cls, sorted(iterable, key=key, reverse=reverse) if iterable else ()
  File "/Users/kvelayutham/opt/anaconda3/envs/py38/lib/python3.8/site-packages/pex/vendor/_vendored/attrs/attr/_make.py", line 1848, in __lt__
    return attrs_to_tuple(self) < attrs_to_tuple(other)
TypeError: '<' not supported between instances of 'VCSArtifact' and 'FileArtifact'
    This is fundamentally because this 
    SortedSet
    admits a union of types that do not support inter-comparison: pex/pex/resolve/locked_resolve.py Lines 253 to 255 in </pantsbuild/pex/commit/adf7e5449562fc819a3e93c0f4f4d6e155284af0|adf7e54> Each of these types just accepts the default 
    @attr.s(frozen=True)
    generated comparison methods, which require an exact type match. pantsbuild/pex
    • 1
    • 1
  • c

    cool-easter-32542

    05/13/2023, 1:38 AM
    #2151 Release 2.1.138 Issue created by jsirois Backlog: ☐ pex fails to build pycryptodome due to filename too long #2087 ☐ Allow system_site_packages=True in Virtualenv.create() #1973 ☐ Cleanup sys.path after 
    __pex__
    is imported #1959 ☐ importing a 
    __pex__
    leaks bootstrap vendored code #1954 ☐ The --compile option is only useful for Python 2.7 #1939 ☐ Universal lock creation failure for disjoint IC and Requires-Python is correct but the error message from Pip / Pex is confusing. #1927 ☐ The 
    __pex__
    import hook does not handle registering top level PEP-420 packages #1910 ☐ Expose setuptools and wheel version flags for use with non-vendored Pips. #1895 ☐ Plumb 
    --pip-version
    to Platform tag calculation. #1894 * * * ☐ PEX_INHERIT_PATH+default shebangs don't interact well with virtualenvs #1344 ☐ Pex interpreter cache is not upgrade / downgrade safe. #1176 ☐ Provide a better resolve error message for a Platform DistributionTarget when 
    --resolve-local-platforms
    . #1070 ☐ Remove 
    PEX_PYTHON_PATH
    from build time interpreter selection. #1075 pantsbuild/pex
  • c

    cool-easter-32542

    05/13/2023, 5:58 PM
    #19002 libcrypt.so.1 can't be found Issue created by jjgalvez Describe the bug Installed pants via script as per instructions, and when I try to run pants in a project folder I get the following error:
    Copy code
    pants
/home/jc/.cache/nce/cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e/cpython-3.9.15+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
Error: Isolates your Pants from the elements.
    Pants version what ever get-pants installs as of 5/13/23 OS Are you encountering the bug on MacOS, Linux, or both? Linux (Arch) Additional info Add any other information about the problem here, such as attachments or links to gists, if relevant. pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/15/2023, 3:31 PM
    #19007 `src/python/pants/pantsd/service/store_gc_service_test` is flaky Issue created by tobni Describe the bug Seemingly intermittently, the test case 
    src/python/pants/pantsd/service/store_gc_service_test.py:tests
    fails with error messages like
    Copy code
    CRITICAL pants.pantsd.service.store_gc_service:store_gc_service.py:85 GC failed: Exception('Garbage collection failed: "Invalid file store entry at \\"/home/runner/.cache/pants/lmdb_store/immutable/files/3f/.tmpyJIvqD\\": InvalidStringLength"')
    Some runs 1. Test Python (Linux-x86_64) Shard 2/3 (3.7) 2. Test Python (Linux-x86_64) Shard 2/3 (3.7) Pants version 2.17.X OS n/a Additional info None pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/15/2023, 5:22 PM
    #18526 Add support for `yarn@v1` for the `javascript` backend Issue created by tobni Is your feature request related to a problem? Please describe. Some users might prefer 
    yarn
    over 
    npm
    , most likely because they're migrating a codebase. Describe the solution you'd like Users can pick a default package manager for the project in the 
    nodejs
    subsystem. Users can also choose a particular package manager for different 
    package_json
    targets (a nodejs resolve should require the same one for all, though). Describe alternatives you've considered Only support npm. Will make some adopters shy away Additional context Blocked on #18525 pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/15/2023, 8:21 PM
    #19011 .gitignored named_caches_dir results in whacky behavior Issue created by cburroughs Describe the bug Under certain conditions setting 
    named_caches_dir
    to a directory that is .gitignored results in python import errors and general whacky behavior. Using 
    named_caches_dir
    within the git repo is sometimes used in CI, where GitLab for example is cranky about caching files outside of what the call the "build root". I've extracted a smallish if not minimal case at: https://github.com/cburroughs/example-python/tree/crazy-cache
    Copy code
    $ pants --no-pantsd --no-local-cache  test ::
<snip>
✓ helloworld/greet/greeting_test.py:tests succeeded in 12.88s.
✓ helloworld/translator/bar_test.py:tests_py310 succeeded in 8.67s.
✓ helloworld/translator/bar_test.py:tests_py37 succeeded in 1.07s.
✓ helloworld/translator/foo_test.py:tests_py310 succeeded in 5.39s.
✓ helloworld/translator/foo_test.py:tests_py37 succeeded in 6.11s.
✓ helloworld/translator/translator_test.py:tests_py310 succeeded in 1.46s.
✓ helloworld/translator/translator_test.py:tests_py37 succeeded in 1.02s.
    ``` $ PANTS_CONFIG_FILES=pants.ci.toml pants --no-pantsd --no-local-cache test :: 155421.95 [INFO] Starting: Building pytest.pex from resource://pants.backend.python.subsystems/pytest.lock 155422.16 [INFO] Starting: Installing python-3.7.lock for the resolve 
    def37
    155422.19 [INFO] Canceled: Building pytest.pex from resource://pants.backend.python.subsystems/pytest.lock 155422.19 [INFO] Starting: Building pytest.pex from resource://pants.backend.python.subsystems/pytest.lock 155422.60 [INFO] Starting: Building pytest.pex from resource://pants.backend.python.subsystems/pytest.lock 155422.84 [INFO] Starting: Installing python-default.lock for the resolve 
    default
    155505.50 [INFO] Completed: Building pytest.pex from resource://pants.backend.python.subsystems/pytest.lock 155506.51 [INFO] Completed: Building pytest.pex from resource://pants.backend.python.subsystems/pytest.lock 155545.28 [INFO] Long running tasks: 82.43s Installing python-default.lock for the resolve 
    default
    83.12s Installing python-3.7.lock for the resolve 
    def37
    155615.34 [INFO] Long running tasks: 112.50s Installing python-default.lock for the resolve 
    default
    113.19s Installing python-3.7.lock for the resolve 
    def37
    155645.39 [INFO] Long running tasks: 142.55s Installing python-default.lock for the resolve 
    default
    143.23s Installing python-3.7.lock for the resolve 
    def37
    155715.45 [INFO] Long running tasks: 172.61s Installing python-default.lock for the resolve 
    default
    173.30s Installing python-3.7.lock for the resolve 
    def37
    155745.50 [INFO] Long running tasks: 202.65s Installing python-default.lock for the resolve 
    default
    203.34s Installing python-3.7.lock for the resolve 
    def37
    155815.53 [INFO] Long running tasks: 232.69s Installing python-default.lock for the resolve 
    default
    233.38s Installing python-3.7.lock for the resolve 
    def37
    155845.59 [INFO] Long running tasks: 262.74s Installing python-default.lock for the resolve 
    default
    263.43s Installing python-3.7.lock for the resolve 
    def37
    155915.66 [INFO] Long running tasks: 292.82s Installing python-default.lock for the resolve 
    default
    293.50s Installing python-3.7.lock for the resolve 
    def37
    155945.69 [INFO] Long running tasks: 322.85s Installing python-default.lock for the resolve 
    default
    323.54s Installing python-3.7.lock for the resolve 
    def37
    160015.73 [INFO] Long running tasks: 352.89s Installing python-default.lock for the resolve 
    default
    353.58s Installing python-3.7.lock for the resolve 
    def37
    160045.78 [INFO] Long running tasks: 382.93s Installing python-default.lock for the resolve 
    default
    383.62s Installing python-3.7.lock for the resolve 
    def37
    160103.04 [INFO] Completed: Installing python-default.lock for the resolve 
    default
    160103.05 [INFO] Starting: Building requirements.pex from python-default.lock 160103.08 [INFO] Canceled: Building requirements.pex from python-default.lock 160103.08 [INFO] Starting: Building requirements.pex from python-default.lock 160115.19 [INFO] Completed: Installing python-3.7.lock for the resolve 
    def37
    160115.20 [INFO] Starting: Building requirements.pex from python-3.7.lock 160115.24 [INFO] Canceled: Building requirements.pex from python-3.7.lock 160116.17 [INFO] Starting: Building requirements.pex from python-3.7.lock 160209.09 [INFO] Completed: Building requirements.pex from python-default.lock 160209.09 [INFO] Starting: Building local_dists.pex 160210.26 [INFO] Completed: Building requirements.pex from python-3.7.lock 160210.27 [INFO] Starting: Building local_dists.pex 160211.12 [INFO] Completed: Building local_dists.pex 160211.17 [INFO] Starting: Building pytest_runner.pex 160224.47 [INFO] Completed: Building local_dists.pex 160224.47 [INFO] Starting: Building pytest_runner.pex 160307.49 [INFO] Completed: Building pytest_runner.pex 160346.39 [INFO] Long running tasks: 81.91s Building pytest_runner.pex 160355.03 [INFO] Completed: Run Pytest - helloworld/translator/bar_test.py:tests_py310 - succeeded. 160355.03 [INFO] Completed: Run Pytest - helloworld/translator/translator_test.py:tests_py310 - succeeded. 160414.52 [INFO] Completed: Run Pytest - helloworld/greet/greeting_test.py:tests - succeeded. 160414.53 [INFO] Completed: Building pytest_runner.pex 160510.93 [ERROR] Completed: Run Pytest - helloworld/translator/foo_test.py:tests_py310 - failed (exit code 2). ============================= test session starts ============================== collecting ... collected 0 items / 1 error ==================================== ERRORS ==================================== ______________ ERROR collecting helloworld/translator/foo_test.py ______________ helloworld/translator/foo_test.py1 in <module> from airflow.models import Variable /home/ecsb/src/o/example-python/.cache/pants/named_caches/pex_root/venvs/s/fd2f0f00/venv/lib/python3.10/site-packages/airflow/__init__.py34 in <module> from airflow import settings /home/ecsb/src/o/example-python/.cache/pants/named_caches/pex_root/venvs/s/fd2f0f00/venv/lib/python3.10/site-packages/airflow/settings.py43 in <module> TIMEZONE = pendulum.tz.timezone('UTC') E AttributeError: module 'pendulum' has no attribute 'tz' =============================== warnings summary =============================== ../../home/ecsb/src/o/example-python/.cache/pants/named_caches/pex_root/venvs/s/fd2f0f00/venv/lib/python3.10/site-packages/airflow/configuration.py:270 /home/ecsb/src/o/example-python/.cache/pants/named_caches/pex_root/venvs/s/fd2f0f00/venv/lib/python3.10/site-packages/airflow/configuration.py270 DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils.version import StrictVersion -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: /tmp/pants-sandbox-om6ajE/helloworld.translator.foo_test.py.tests_py310.xml - =========================== short test summary info ============================ ERROR helloworld/translator/foo_test.py - AttributeError: module 'pendulum' h... !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!! ========================= 1 warning, 1 error in 5.42s ========================== 160513.54 [INFO] Completed: Run Pytest - helloworld/translator/translator_test.py:tests_py37 - succeeded. 160513.68 [INFO] Completed: Run Pytest - helloworld/translator/bar_test.py:tests_py37 - succeeded. 160516.53 [INFO] Long running tasks: 61.98s Run Pytest for helloworld/translator/foo_test.py:tests_py37 160516.92 [INFO] Completed: Run Pytest - helloworld/translator/foo_test.py:tests_py37 - succeeded. ✓ helloworld/greet/greeting_test.py:tests succeeded in 19.68s. ✓ helloworld/translator/bar_test.py:tests_py… pantsbuild/pants
  • c

    cool-easter-32542

    05/16/2023, 2:50 AM
    #19013 `system_binary` default fingerprinting behaviour (run without args) fails for many programs and is hard to debug Issue created by huonw Describe the bug 
    system_binary
    is nifty in 2.16, and it supports running the underlying program to 'fingerprint' it. For the simple case of no explicit 
    fingerprint_...
    args, this translates to running it without any args, e.g. 
    system_binary(name="git", binary_name="git")
    will run 
    git
    . This sort of invocation returns non-zero (fails) for many programs, like 
    git
    , 
    npm
    , even 
    sed
    on some systems. The pants output doesn't make it at all obvious what the problem is. Reproducer:
    Copy code
    cd $(mktemp -d)

cat > pants.toml <<EOF
[GLOBAL]
pants_version = "2.16.0rc2"
backend_packages = ["pants.backend.experimental.adhoc"]

[anonymous-telemetry]
enabled = false
EOF

cat > BUILD <<EOF
system_binary(name="git-fingerprintless", binary_name="git")
adhoc_tool(name="fingerprintless", runnable=":git-fingerprintless", args=["--version"])

system_binary(name="git-fingerprinted", binary_name="git", fingerprint_args=["--version"])
adhoc_tool(name="fingerprinted", runnable=":git-fingerprinted", args=["--version"])
EOF

echo "Shell finds git at: $(which git)"

# OK: this works
pants export-codegen :fingerprinted

# BUG: this fails non-obviously
pants export-codegen :fingerprintless
    Lightly edited output of 
    bash ./script.sh
    Copy code
    Shell finds git at: /opt/homebrew/bin/git


12:46:46.55 [INFO] Initializing scheduler...
12:46:46.73 [INFO] Scheduler initialized.
12:46:46.97 [INFO] Starting: Testing candidate for `git` at `/usr/bin/git`
12:46:46.97 [INFO] Starting: Testing candidate for `git` at `/opt/homebrew/bin/git`
12:46:47.00 [INFO] Completed: Testing candidate for `git` at `/opt/homebrew/bin/git`
12:46:47.00 [INFO] Completed: Testing candidate for `git` at `/usr/bin/git`
12:46:47.00 [INFO] Starting: Running the `adhoc_tool` at //:fingerprinted
12:46:47.01 [INFO] Completed: Running the `adhoc_tool` at //:fingerprinted
12:46:47.02 [INFO] Writing generated files to dist/codegen


12:46:47.44 [INFO] Starting: Testing candidate for `git` at `/usr/bin/git`
12:46:47.44 [INFO] Starting: Testing candidate for `git` at `/opt/homebrew/bin/git`
12:46:47.45 [INFO] Completed: Testing candidate for `git` at `/opt/homebrew/bin/git`
12:46:47.45 [INFO] Completed: Testing candidate for `git` at `/usr/bin/git`
12:46:47.45 [ERROR] 1 Exception encountered:

Engine traceback:
  in `export-codegen` goal

ValueError: Could not find a binary with name `git`. The following paths were searched: /usr/bin, /bin, /usr/local/bin, /opt/homebrew/bin.
    This is confusing for users as the output explicitly says "could not find a binary", despite them several of them clearly existing. In addition, the docs are somewhat ambiguous, potentially suggesting fingerprinting may only happen if 
    fingerprint
    is specified:
    Pants will search for binaries with name binary_name in the search paths provided, as well as default search paths. If 
    fingerprint
    is specified, each binary that is located will be executed with the arguments from 
    fingerprint_args
    . Any binaries whose output does not match the pattern will be excluded.
    Some (not mutually exclusive) options for improving this: 1. tweak the failure message to be something like "Could not find an appropriate binary with name `git`" or something 2. if all candidates are rejected, include the reason for failure for each candidate that exists and was rejected (e.g. exit code and/or command output), and maybe a suggestion related to tweaking the 
    fingerprint...
    config 3. tweak the behaviour of the fingerprinting itself e.g. 1. don't do any fingerprinting when there's no fingerprint args (just test for file existence) 2. have those args be required, e.g. instead of the implicit 
    fingerprint_args=[]
    , the user has to write that The backend in question is still experimental. Pants version 2.16.0rc2 OS macOS Additional info Encountered in practice in https://pantsbuild.slack.com/archives/C046T6T9U/p1684197510244099 (
    git
    ) and 329da41 (
    sed
    ) pantsbuild/pants
  • c

    cool-easter-32542

    05/16/2023, 4:58 PM
  • c

    cool-easter-32542

    05/16/2023, 6:51 PM
    #19017 `AdhocProcessRequest` isn't replacing `$0` correctly Issue created by thejcannon Describe the bug From https://pantsbuild.slack.com/archives/C0D7TNJHL/p1684255827991839
    Copy code
    file(
    name="downloaded-jq",
    source = http_source(
        len = 3953824,
        sha256 = "af986793a515d500ab2d35f8d2aecd656e764504b789b66d7e1a0b727a124c44",
        url = "<https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64>",
    ),
)
shell_command(
    name = "sandboxed-jq",
    command="chmod +x jq-linux64",
    tools = ["chmod"],
    execution_dependencies=[":downloaded-jq"],
    output_directories = ["."],
)
run_shell_command(
    name = "jq",
    command=f"{{chroot}}/{build_file_dir() / 'jq-linux64'}",
    execution_dependencies=[":sandboxed-jq"],
    workdir="/",
)
    Run with: 
    $ pants run //:jq -- --help
    yields usage:
    Copy code
    Usage:  /tmp/pants-sandbox-zLIAv5/3rdparty/tools/jq-linux64 [options] <jq filter> [file...]
        /tmp/pants-sandbox-zLIAv5/3rdparty/tools/jq-linux64 [options] --args <jq filter> [strings...]
        /tmp/pants-sandbox-zLIAv5/3rdparty/tools/jq-linux64 [options] --jsonargs <jq filter> [JSON_TEXTS...]
    Pants version 
    main
    OS Linux Additional info pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/16/2023, 7:51 PM
    #19018 Have `--keep-sandboxes` not restart daemon Issue created by thejcannon Is your feature request related to a problem? Please describe. If I want to preserve a sandbox that passes I have to use 
    --keep-sandboxes=always
    , but that restarts the daemon, meaning I'll preserve many more sandboxes than what I wanted. Describe the solution you'd like Changing 
    --keep-sandboxes
    doesn't invalidate the daemon. Describe alternatives you've considered N/A Additional context In DM @witty-crayon-22786 said:
    With a bunch of refactoring it could. Currently it's a constructor parameter for all of the CommandRunners, but you could probably move it to the process::Context.
    pantsbuild/pants
  • c

    cool-easter-32542

    05/16/2023, 8:35 PM
    #15315 `poetry&lt;=1.2.0b1` does not distinguish between `alpha` and `dev` releases Issue created by stuhood Attempting to consume 
    2.12.0a0
    using 
    pants_requirements
    in a repository using 
    [python].lockfile_generator=poetry
    will hit python-poetry/poetry#4653, which causes 
    poetry
    to create a lockfile containing 
    2.12.0.dev3
    , which then cannot actually be consumed. • Temporarily switching to PEX for lockfile generation 1) only for the relevant resolve and 2) only in this release... seems to work: i.e.: 
    ./pants --python-lockfile-generator=pex generate-lockfiles --resolve=pants-plugin
    . • Using an explicit 
    python_requirement
    target rather than 
    pants_requirements
    does not work: 
    poetry
    converts the 
    pantsbuild.pants==2.12.0a0
    requirement into 
    2.12.0dev0
    (whee). • Upgrading 
    poetry
    to 
    1.2.0*
    would probably fix this... although an end user is not able to do this, because they'll encounter an incompatibility in our shim. pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/16/2023, 10:05 PM
    #2154 Pex intermittently mishandling transitive dependencies Issue created by tylerscheuble Building a pex with 
    sentry-sdk==1.23.0
    fails around half the time with the following error:
    Copy code
    Failed to resolve compatible distributions:
1: sentry-sdk==1.23.0 requires urllib3<2.0.0 but urllib3 2.0.2 was resolved
    Screenshot:

    image▾

    pantsbuild/pex
    • 1
    • 1
  • c

    cool-easter-32542

    05/18/2023, 3:25 PM
    Comment on #18981 pants does not find a wheel Discussion answered by jsirois No, 
    cp311-none-any
    is not the catch all. Basically cp implies platfrom specific and you'll only find it in platform-specific tags. The catch all family is 
    py311-none-any
    . The answer to all this lies in the PyPA 
    packaging
    library. Pex uses this and you can see what Pex sees (via 
    packaging.tags
    ) with the following tool (available when you 
    pip install pex
    in a venv. N.B.: That the important variance you should see is a different manylinux support level (glibc). Mine is 2.35 and older. One wheel variant you showed above was 2.28 and older (
    3rdparty/repos/orekit-10.3.1-cp311-cp311-manylinux_2_28_x86_64.whl
    ). Also N.B.: You can specify the full path to the exact Python interpreter you mean with 
    --python /the/full/path/to/python
    .
    Copy code
    $ pex3 interpreter inspect --python python3.11 -t -i2
    ``` { "path": "/home/jsirois/dev/pantsbuild/jsirois-pex/.tox/py311-integration/bin/python", "compatible_tags": [ "cp311-cp311-manylinux_2_35_x86_64", "cp311-cp311-manylinux_2_34_x86_64", "cp311-cp311-manylinux_2_33_x86_64", "cp311-cp311-manylinux_2_32_x86_64", "cp311-cp311-manylinux_2_31_x86_64", "cp311-cp311-manylinux_2_30_x86_64", "cp311-cp311-manylinux_2_29_x86_64", "cp311-cp311-manylinux_2_28_x86_64", "cp311-cp311-manylinux_2_27_x86_64", "cp311-cp311-manylinux_2_26_x86_64", "cp311-cp311-manylinux_2_25_x86_64", "cp311-cp311-manylinux_2_24_x86_64", "cp311-cp311-manylinux_2_23_x86_64", "cp311-cp311-manylinux_2_22_x86_64", "cp311-cp311-manylinux_2_21_x86_64", "cp311-cp311-manylinux_2_20_x86_64", "cp311-cp311-manylinux_2_19_x86_64", "cp311-cp311-manylinux_2_18_x86_64", "cp311-cp311-manylinux_2_17_x86_64", "cp311-cp311-manylinux2014_x86_64", "cp311-cp311-manylinux_2_16_x86_64", "cp311-cp311-manylinux_2_15_x86_64", "cp311-cp311-manylinux_2_14_x86_64", "cp311-cp311-manylinux_2_13_x86_64", "cp311-cp311-manylinux_2_12_x86_64", "cp311-cp311-manylinux2010_x86_64", "cp311-cp311-manylinux_2_11_x86_64", "cp311-cp311-manylinux_2_10_x86_64", "cp311-cp311-manylinux_2_9_x86_64", "cp311-cp311-manylinux_2_8_x86_64", "cp311-cp311-manylinux_2_7_x86_64", "cp311-cp311-manylinux_2_6_x86_64", "cp311-cp311-manylinux_2_5_x86_64", "cp311-cp311-manylinux1_x86_64", "cp311-cp311-linux_x86_64", "cp311-abi3-manylinux_2_35_x86_64", "cp311-abi3-manylinux_2_34_x86_64", "cp311-abi3-manylinux_2_33_x86_64", "cp311-abi3-manylinux_2_32_x86_64", "cp311-abi3-manylinux_2_31_x86_64", "cp311-abi3-manylinux_2_30_x86_64", "cp311-abi3-manylinux_2_29_x86_64", "cp311-abi3-manylinux_2_28_x86_64", "cp311-abi3-manylinux_2_27_x86_64", "cp311-abi3-manylinux_2_26_x86_64", "cp311-abi3-manylinux_2_25_x86_64", "cp311-abi3-manylinux_2_24_x86_64", "cp311-abi3-manylinux_2_23_x86_64", "cp311-abi3-manylinux_2_22_x86_64", "cp311-abi3-manylinux_2_21_x86_64", "cp311-abi3-manylinux_2_20_x86_64", "cp311-abi3-manylinux_2_19_x86_64", "cp311-abi3-manylinux_2_18_x86_64", "cp311-abi3-manylinux_2_17_x86_64", "cp311-abi3-manylinux2014_x86_64", "cp311-abi3-manylinux_2_16_x86_64", "cp311-abi3-manylinux_2_15_x86_64", "cp311-abi3-manylinux_2_14_x86_64", "cp311-abi3-manylinux_2_13_x86_64", "cp311-abi3-manylinux_2_12_x86_64", "cp311-abi3-manylinux2010_x86_64", "cp311-abi3-manylinux_2_11_x86_64", "cp311-abi3-manylinux_2_10_x86_64", "cp311-abi3-manylinux_2_9_x86_64", "cp311-abi3-manylinux_2_8_x86_64", "cp311-abi3-manylinux_2_7_x86_64", "cp311-abi3-manylinux_2_6_x86_64", "cp311-abi3-manylinux_2_5_x86_64", "cp311-abi3-manylinux1_x86_64", "cp311-abi3-linux_x86_64", "cp311-none-manylinux_2_35_x86_64", "cp311-none-manylinux_2_34_x86_64", "cp311-none-manylinux_2_33_x86_64", "cp311-none-manylinux_2_32_x86_64", "cp311-none-manylinux_2_31_x86_64", "cp311-none-manylinux_2_30_x86_64", "cp311-none-manylinux_2_29_x86_64", "cp311-none-manylinux_2_28_x86_64", "cp311-none-manylinux_2_27_x86_64", "cp311-none-manylinux_2_26_x86_64", "cp311-none-manylinux_2_25_x86_64", "cp311-none-manylinux_2_24_x86_64", "cp311-none-manylinux_2_23_x86_64", "cp311-none-manylinux_2_22_x86_64", "cp311-none-manylinux_2_21_x86_64", "cp311-none-manylinux_2_20_x86_64", "cp311-none-manylinux_2_19_x86_64", "cp311-none-manylinux_2_18_x86_64", "cp311-none-manylinux_2_17_x86_64", "cp311-none-manylinux2014_x86_64", "cp311-none-manylinux_2_16_x86_64", "cp311-none-manylinux_2_15_x86_64", "cp311-none-manylinux_2_14_x86_64", "cp311-none-manylinux_2_13_x86_64", "cp311-none-manylinux_2_12_x86_64", "cp311-none-manylinux2010_x86_64", "cp311-none-manylinux_2_11_x86_64", "cp311-none-manylinux_2_10_x86_64", "cp311-none-manylinux_2_9_x86_64", "cp311-none-manylinux_2_8_x86_64", "cp311-none-manylinux_2_7_x86_64", "cp311-none-manylinux_2_6_x86_64", "cp311-none-manylinux_2_5_x86_64", "cp311-none-manylinux1_x86_64", "cp311-none-linux_x86_64", "cp310-abi3-manylinux_2_35_x86_64", "cp310-abi3-manylinux_2_34_x86_64", "cp310-abi3-manylinux_2_33_x86_64", "cp310-abi3-manylinux_2_32_x86_64", "cp310-abi3-manylinux_2_31_x86_64", "cp310-abi3-manylinux_2_30_x86_64", "cp310-abi3-manylinux_2_29_x86_64", "cp310-abi3-manylinux_2_28_x86_64", "cp310-abi3-manylinux_2_27_x86_64", "cp310-abi3-manylinux_2_26_x86_64", "cp310-abi3-manylinux_2_25_x86_64", "cp310-abi3-manylinux_2_24_x86_64", "cp310-abi3-manylinux_2_23_x86_64", "cp310-abi3-manylinux_2_22_x86_64", "cp310-abi3-manylinux_2_21_x86_64", "cp310-abi3-manylinux_2_20_x86_64", "cp310-abi3-manylinux_2_19_x86_64", "cp310-abi3-manylinux_2_18_x86_64", "cp310-abi3-manylinux_2_17_x86_64", "cp310-abi3-manylinux2014_x86_64", "cp310-abi3-manylinux_2_16_x86_64", "cp310-abi3-manylinux_2_15_x86_64", "cp310-abi3-manylinux_2_14_x86_64", "cp310-abi3-manylinux_2_13_x86_64", "cp310-abi3-manylinux_2_12_x86_64", "cp310-abi3-manylinux2010_x86_64", "cp310-abi3-manylinux_2_11_x86_64", "cp310-abi3-manylinux_2_10_x86_64", "cp310-abi3-manylinux_2_9_x86_64", "cp310-abi3-manylinux_2_8_x86_64", "cp310-abi3-manylinux_2_7_x86_64", "cp310-abi3-manylinux_2_6_x86_64", "cp310-abi3-manylinux_2_5_x86_64", "cp310-abi3-manylinux1_x86_64", "cp310-abi3-linux_x86_64", "cp39-abi3-manylinux_2_35_x86_64", "cp39-abi3-manylinux_2_34_x86_64", "cp39-abi3-manylinux_2_33_x86_64", "cp39-abi3-manylinux_2_32_x86_64", "cp39-abi3-manylinux_2_31_x86_64", "cp39-abi3-manylinux_2_30_x86_64", "cp39-abi3-manylinux_2_29_x86_64", "cp39-abi3-manylinux_2_28_x86_64", "cp39-abi3-manylinux_2_27_x86_64", "cp39-abi3-manylinux_2_26_x86_64", "cp39-abi3-manylinux_2_25_x86_64", "cp39-abi3-manylinux_2_24_x86_64", "cp39-abi3-manylinux_2_23_x86_64", "cp39-abi3-manylinux_2_22_x86_64", "cp39-abi3-manylinux_2_21_x86_64", "cp39-abi3-manylinux_2_20_x86_64", "cp39-abi3-manylinux_2_19_x86_64", "cp39-abi3-manylinux_2_18_x86_64", "cp39-abi3-manylinux_2_17_x86_64", "cp39-abi3-manylinux2014_x86_64", "cp39-abi3-manylinux_2_16_x86_64", "cp39-abi3-manylinux_2_15_x86_64", "cp39-abi3-manylinux_2_14_x86_64", "cp39-abi3-manylinux_2_13_x86_64", "cp39-abi3-manylinux_2_12_x86_64", "cp39-abi3-manylinux2010_x86_64", "cp39-abi3-manylinux_2_11_x86_64", "cp39-abi3-manylinux_2_10_x86_64", "cp39-abi3-manylinux_2_9_x86_64", "cp39-abi3-manylinux_2_8_x86_64", "cp39-abi3-manylinux_2_7_x86_64", "cp39-abi3-manylinux_2_6_x86_64", "cp39-abi3-manylinux_2_5_x86_64", "cp39-abi3-manylinux1_x86_64", "cp39-abi3-linux_x86_64", "cp38-abi3-manylinux_2_35_x86_64", "cp38-abi3-manylinux_2_34_x86_64", "cp38-abi3-manylinux_2_33_x86_64", "cp38-abi3-manylinux_2_32_x86_64", "cp38-abi… pantsbuild/pants
  • c

    cool-easter-32542

    05/18/2023, 3:41 PM
    #19035 how to generate requirements.txt from BUILD file? New discussion created by jessehersch Hi pants experts! Anyone know how to generate a python requirements.txt file from all the libs in 3rdparty/BUILD? I am using pants 2.15.0 in pants 1.3.0 (ancient I know 😊) I used to do this.
    Copy code
    ./pants dependencies2 --dependencies2-type=3rdparty :: | sort -u > ./requirements.txt
    In 2.15 there does not seem to be an equivalent of the type arg:
    Copy code
    `dependencies` goal options
---------------------------

List the dependencies of the input files/targets.
 
Activated by pants.backend.project_info
Config section: [dependencies]
 
  --dependencies-output-file=<path>
  PANTS_DEPENDENCIES_OUTPUT_FILE
  output_file
      default: None
      current value: None
      Output the goal's stdout to this file. If unspecified, outputs to stdout.

  --dependencies-sep=<separator>
  PANTS_DEPENDENCIES_SEP
  sep
      default: \n
      current value: \n
      String to use to separate lines in line-oriented output.

  --[no-]dependencies-transitive
  PANTS_DEPENDENCIES_TRANSITIVE
  transitive
      default: False
      current value: False
      List all transitive dependencies. If unspecified, list direct dependencies only.

  --[no-]dependencies-closed
  PANTS_DEPENDENCIES_CLOSED
  closed
      default: False
      current value: False
      Include the input targets in the output, along with the dependencies.
    Jesse pantsbuild/pants
  • c

    cool-easter-32542

    05/18/2023, 5:12 PM
    #18625 Using `[mypy].source_plugins` with `install_from_resolve` may be unergonomic Issue created by kaos Describe the inconvenience When running 
    pants check
    with 
    mypy
    configured to run off of a regular lockfile using 
    install_from_resolve
    any 3rdparty dependencies required for code referenced with 
    source_plugins
    must be explicitly listed in 
    [mypy].requirements
    . Pants version 2.16.0rc1 OS Mac Additional info https://www.pantsbuild.org/v2.16/docs/reference-mypy#source_plugins pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/18/2023, 5:14 PM
    #9417 nailgun-server dependency not found when PANTS_WORKDIR not under cwd Issue created by bjorn-spire I have been updating our repository to use a newer version of pants (we have been on 1.16) and I saw the warnings about Ivy going away, and while we don't use JVM at this point, I figured I should silence the warning and did the upgrade step. But then we couldn't run pants on our CI servers anymore. One of the things we have done on CI to keep things a bit faster on each node is to set 
    export PANTS_WORKDIR=/var/go/ais-pipeline.pants.d
    (go is our CI system, nothing to do with the language) and I noticed that when running on CI if I didn't have this environment variable set it resolves correctly. For now I have resolved this by removing the setting for Coursier, but I would love to have this keep working going forward. I would be happy to have a go at trying to fix this with some guidance. I am guessing that the hint with cwd in the log below may make it tougher than I think? :x The errors we got when trying to run 
    ./pants -ldebug test
    was:
    Copy code
    […snip…]
06:33:44 [DEBUG] pants.binaries.binary_util:pid=19352: binary_request: BinaryRequest(supportdir='bin/coursier', version='1.1.0.cf365ea27a710d5f09db1f0a6feee129aa1fc417', name='coursier', platform_dependent=False, external_url_generator=<pants.backend.jvm.tasks.coursier.coursier_subsystem.CoursierUrlGenerator object at 0x7e068df512b0>, archiver=None)
06:33:44 [DEBUG] pants.binaries.binary_util:pid=19352: self._allow_external_binary_tool_downloads: True
06:33:44 [DEBUG] pants.binaries.binary_util:pid=19352: external_url_generator: <pants.backend.jvm.tasks.coursier.coursier_subsystem.CoursierUrlGenerator object at 0x7e068df512b0>
06:33:44 [DEBUG] pants.binaries.binary_util:pid=19352: fetch_request: BinaryFetchRequest(download_path='bin/coursier/1.1.0.cf365ea27a710d5f09db1f0a6feee129aa1fc417/coursier', urls=('<https://github.com/coursier/coursier/releases/download/pants_release_1.5.x/coursier-cli-1.1.0.cf365ea27a710d5f09db1f0a6feee129aa1fc417.jar>',))
06:33:44 [DEBUG] pants.binaries.binary_util:pid=19352: bootstrapped_binary_path: /var/go/.cache/pants/bin/coursier/1.1.0.cf365ea27a710d5f09db1f0a6feee129aa1fc417/coursier
06:33:44 [DEBUG] pants.binaries.binary_util:pid=19352: Selected coursier binary bootstrapped to: /var/go/.cache/pants/bin/coursier/1.1.0.cf365ea27a710d5f09db1f0a6feee129aa1fc417/coursier

06:33:44 00:11       [coursier]06:33:44 [DEBUG] pants.java.executor:pid=19352: Executing: /usr/bin/java -Xmx256m -cp /var/go/.cache/pants/bin/coursier/1.1.0.cf365ea27a710d5f09db1f0a6feee129aa1fc417/coursier coursier.cli.Coursier fetch -t --cache /var/go/.cache/pants/coursier -r <https://repo1.maven.org/maven2> -A jar,bundle,test-jar,maven-plugin,src,doc -q --no-default -n 8 --json-output-file /var/go/ais-pipeline.pants.d/bootstrap/bootstrap-jvm-tools/a0ebe8e0b001/tmp/tmpxgduw_td com.martiansoftware:nailgun-server:0.9.1 args={} at cwd=/var/lib/go-agent1/pipelines/ais-pipeline--pr
06:33:49 [DEBUG] pants.process.lock:pid=19352: releasing lock: <pants.process.lock.OwnerPrintingInterProcessFileLock object at 0x7e06941f2828>

               Waiting for background workers to finish.
06:33:49 00:16   [complete]
               FAILURE

ERROR: /var/lib/go-agent1/pipelines/ais-pipeline--pr is not a directory containing /var/go/ais-pipeline.pants.d/bootstrap/bootstrap-jvm-tools/a0ebe8e0b001/cache/relative/https/repo1.maven.org/maven2/com/martiansoftware/nailgun-server/0.9.1/nailgun-server-0.9.1.jar
    The file in question exists:
    Copy code
    $ ls -l /var/go/ais-pipeline.pants.d/bootstrap/bootstrap-jvm-tools/a0ebe8e0b001/cache/relative/https/repo1.maven.org/maven2/com/martiansoftware/nailgun-server/0.9.1/nailgun-server-0.9.1.jar
-rw-r--r--. 2 go go 32927 Dec  3  2012 /var/go/ais-pipeline.pants.d/bootstrap/bootstrap-jvm-tools/a0ebe8e0b001/cache/relative/https/repo1.maven.org/maven2/com/martiansoftware/nailgun-server/0.9.1/nailgun-server-0.9.1.jar
    pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/18/2023, 5:22 PM
    #12397 Pants should be distributed on Github Releases Issue created by stuhood There are two large problems with our use of PyPI: 1. The https://github.com/pantsbuild/setup script resolves using 
    PIP
    for the specified 
    pantsbuild.pants
    wheel version, but without using a lockfile. (relates to Verifying Pants installations) 2. We have hit PyPI's size limits in the past, and are close to doing so again. See #11614. * * * To ensure that using a particular version of Pants is reproducible over time, and to reduce load on PyPI, we should either: 1. distribute Pants as a pre-built PEX 2. distribute Pants as a lockfile that fully pins transitive dependencies for supported platforms (with the same considerations as #12200) • EDIT: Does not resolve #11614. 3. have the 
    pants
    script dynamically generate and recommend committing a lockfile     EDIT: Does not resolve #11614. 4. switch the distribution model of Pants to a native binary • EDIT: #7369 was resolved without removing use of PyPI. • It's still possible that we could actually use PyOxidizer or "a 
    scie
    containing a PEX and an interpreter" to release a Pants binary, at which point 
    scie-pants
    would not need to bootstrap an interpreter. pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/18/2023, 10:43 PM
    #19046 What does 247 exit code mean? New discussion created by tobiasjcat Hey, I'm running pants 2.15.0 to run a set of unittest cases but I keep getting this error only when running the code with pants. When I run the test cases locally with pytest they succeed.
    Copy code
    ✕ <dir>/<dir>/test_<name>.py:test failed in 1280.49s.


Process exited with code 247
    What does this mean? How can I address it? pantsbuild/pants
  • c

    cool-easter-32542

    05/19/2023, 3:36 AM
    #19049 Upload large files directly from disk, without loading into memory Issue created by huonw Is your feature request related to a problem? Please describe. With #18153, large files produced are now cached as standalone files, on disk, rather than in the core LMDB structures. Uploading to a remote cache should be able to pipe these directly from that location on disk. Currently they're pulled into memory, written to a temporary file (with an additional bug: it's sync IO in async code), and then that temporary file is uploaded via mmap. Breadcrumbs, in `src/rust/engine/fs/store`: 1. 
    store::Store::ensure_remote_has_recursive
    calls 
    store::Store::store_large_blob_remote
    2. 
    store_large_blob_remote
    calls 
    store::remote::ByteStore::store_buffered
    3. 
    store_buffered
    creates temporary files... 4. ...then calls the closure that 
    store_large_blob_remote
    provided with a 
    std::fs::File
    for that temporary file 5. that closure then reads the whole large-file into memory and splats it into the temporary file (synchronously) Describe the solution you'd like 
    store::Store::ensure_remote_has_recursive
    should pass a file handle into the remote cache, and that's manipulated/uploaded directly, without going through memory. The following code could be adjusted to just decide whether the blob is on disk (if so, go via the disk) or is in LMDB (if so, just pull into memory), resolving the TODO by removing the linking to wire chunk size: pants/src/rust/engine/fs/store/src/lib.rs Lines 831 to 839 in </pantsbuild/pants/commit/4c708a001b19813fdc1fdbcae6ab7d2010208443|4c708a0> Describe alternatives you've considered N/A Additional context Add any other context or screenshots about the feature request here. pantsbuild/pants
  • c

    cool-easter-32542

    05/19/2023, 7:20 AM
    #19051 Pants rejects named cache names that contain digits Issue created by alonsodomin Describe the bug When launching a 
    Process
    with a named cache (i.e. 
    append_only_caches
    ), if any of the cache names contains digits (i.e. 
    namedcache2
    ), Pants will fail with following exception:
    Copy code
    native_engine.IntrinsicError: Error lifting Process: Cache names may only contain lowercase alphanumeric characters or underscores: got "namedcache2"
    The error message is puzzling because it gives the understanding that the given name (one with lowercase alphabetic characters and one digit) meets the criteria. The culprit is the following check in `named_caches.rs`:
    Copy code
    name.chars().all(|c| (c.is_ascii_alphanumeric() && c.is_ascii_lowercase()) || c == '_')
    This is because digits will pass the 
    c.is_ascii_alphanumeric()
    check but will fail at 
    c.is_ascii_lowercase()
    . Pants version This was found in 
    HEAD
    OS both Additional info pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/19/2023, 10:48 AM
    #19053 Unexpectedly high pantsd memory usage with Go Issue created by jyggen We have a relatively small amount of Go code in our monorepository (49 files and 8041 lines according to 
    pants count-loc
    , compared to 879 files and 101251 lines Python), and yet when enabling the Go backend we need to bump 
    pantsd_max_memory_usage
    from the default value up to at least 
    8GiB
    in order to make pantsd not restart on every Pants execution (though it still restarts frequently enough for our Python developers to simply disable the Go backend locally due to the performance degradation it causes). Here's the memory summary after running 
    pants --stats-memory-summary fix lint check test golang/::
    on 2.16.0rc2: ``` 123416.78 [INFO] Memory summary (total size in bytes, count, name): 48 1 abc.EnvironmentAware 48 1 abc.EnvironmentAware 48 1 pants.backend.codegen.protobuf.go.rules.ProtobufGoModuleImportPathsMappingsHook 48 1 pants.backend.codegen.protobuf.go.rules._SetupGoProtocPlugin 48 1 pants.backend.codegen.protobuf.lint.buf.subsystem.BufSubsystem 48 1 pants.backend.codegen.protobuf.protobuf_dependency_inference.ProtobufMapping 48 1 pants.backend.codegen.protobuf.protoc.Protoc 48 1 pants.backend.codegen.protobuf.python.python_protobuf_module_mapper.PythonProtobufMappingMarker 48 1 pants.backend.codegen.protobuf.python.python_protobuf_subsystem.PythonProtobufSubsystem 48 1 pants.backend.docker.lint.hadolint.subsystem.Hadolint 48 1 pants.backend.docker.subsystems.dockerfile_parser.DockerfileParser 48 1 pants.backend.docker.subsystems.dockerfile_parser.ParserSetup 48 1 pants.backend.go.dependency_inference.AllGoModuleImportPathsMappings 48 1 pants.backend.go.dependency_inference.GoModuleImportPathsMapping 48 1 pants.backend.go.goals.check.GoCheckRequest 48 1 pants.backend.go.lint.gofmt.subsystem.GofmtSubsystem 48 1 pants.backend.go.subsystems.golang.GolangSubsystem 48 1 pants.backend.go.subsystems.golang.GolangSubsystem.EnvironmentAware 48 1 pants.backend.go.subsystems.gotest.GoTestSubsystem 48 1 pants.backend.go.target_type_rules.FirstPartyGoModuleImportPathsMappingsHook 48 1 pants.backend.go.target_type_rules.GenerateTargetsFromGoModRequest 48 1 pants.backend.go.target_type_rules.GoImportPathMappingRequest 48 1 pants.backend.go.util_rules.build_pkg.SetupAsmCheckBinary 48 1 pants.backend.go.util_rules.build_pkg_target._ResolveStdlibEmbedConfigRequest 48 1 pants.backend.go.util_rules.build_pkg_target._ResolveStdlibEmbedConfigResult 48 1 pants.backend.go.util_rules.cgo.CGoCompilerWrapperScript 48 1 pants.backend.go.util_rules.cgo.CGoCompilerWrapperScriptRequest 48 1 pants.backend.go.util_rules.cgo_binaries.CGoBinaryPathRequest 48 1 pants.backend.go.util_rules.go_bootstrap.GoBootstrap 48 1 pants.backend.go.util_rules.goroot.GoRoot 48 1 pants.backend.go.util_rules.link.LinkerSetup 48 1 pants.backend.go.util_rules.pkg_analyzer.PackageAnalyzerSetup 48 1 pants.backend.go.util_rules.sdk.GoSdkRunSetup 48 1 pants.backend.go.util_rules.third_party_pkg.AllThirdPartyPackages 48 1 pants.backend.go.util_rules.third_party_pkg.AllThirdPartyPackagesRequest 48 1 pants.backend.go.util_rules.third_party_pkg.ModuleDescriptors 48 1 pants.backend.go.util_rules.third_party_pkg.ModuleDescriptorsRequest 48 1 pants.backend.project_info.filter_targets.FilterSubsystem 48 1 pants.backend.project_info.regex_lint.RegexLintSubsystem 48 1 pants.backend.python.dependency_inference.module_mapper.AllPythonTargets 48 1 pants.backend.python.dependency_inference.module_mapper.FirstPartyPythonModuleMapping 48 1 pants.backend.python.dependency_inference.module_mapper.FirstPartyPythonTargetsMappingMarker 48 1 pants.backend.python.dependency_inference.module_mapper.ThirdPartyPythonModuleMapping 48 1 pants.backend.python.dependency_inference.parse_python_dependencies.GeneralPythonDependencyVisitorRequest 48 1 pants.backend.python.dependency_inference.parse_python_dependencies.ParserScript 48 1 pants.backend.python.dependency_inference.parse_python_dependencies.PythonDependencyVisitor 48 1 pants.backend.python.dependency_inference.subsystem.PythonInferSubsystem 48 1 pants.backend.python.goals.lockfile.PythonSyntheticLockfileTargetsRequest 48 1 pants.backend.python.lint.add_trailing_comma.subsystem.AddTrailingComma 48 1 pants.backend.python.lint.autoflake.subsystem.Autoflake 48 1 pants.backend.python.lint.bandit.subsystem.Bandit 48 1 pants.backend.python.lint.black.subsystem.Black 48 1 pants.backend.python.lint.docformatter.subsystem.Docformatter 48 1 pants.backend.python.lint.flake8.subsystem.Flake8 48 1 pants.backend.python.lint.flake8.subsystem.Flake8FirstPartyPlugins 48 1 pants.backend.python.lint.isort.subsystem.Isort 48 1 pants.backend.python.macros.python_requirements.GenerateFromPythonRequirementsRequest 48 1 pants.backend.python.subsystems.python_native_code.PythonNativeCodeSubsystem 48 1 pants.backend.python.subsystems.python_native_code.PythonNativeCodeSubsystem.EnvironmentAware 48 1 pants.backend.python.subsystems.repos.PythonRepos 48 1 pants.backend.python.subsystems.setup.PythonSetup 48 1 pants.backend.python.target_types_rules.PythonResolveFieldDefaultFactoryRequest 48 1 pants.backend.python.util_rules.interpreter_constraints.InterpreterConstraints 48 1 pants.backend.python.util_rules.pex_cli.PexCli 48 1 pants.backend.python.util_rules.pex_cli.PexPEX 48 1 pants.backend.python.util_rules.pex_environment.PexEnvironment 48 1 pants.backend.python.util_rules.pex_environment.PexSubsystem 48 1 pants.backend.python.util_rules.pex_environment.PexSubsystem.EnvironmentAware 48 1 pants.backend.python.util_rules.pex_environment.PythonExecutable 48 1 pants.backend.python.util_rules.pex_requirements.Lockfile 48 1 pants.backend.python.util_rules.pex_requirements.Resolve 48 1 pants.backend.python.util_rules.python_sources.PythonSourceFiles 48 1 pants.backend.python.util_rules.python_sources.StrippedPythonSourceFiles 48 1 pants.backend.shell.lint.shellcheck.subsystem.Shellcheck 48 1 pants.backend.shell.lint.shfmt.subsystem.Shfmt 48 1 pants.backend.shell.subsystems.shell_tes… pantsbuild/pants
  • c

    cool-easter-32542

    05/19/2023, 10:54 AM
    #19054 javascript: node_modules should be immutable_input_digests Issue created by tobni Is your feature request related to a problem? Please describe. node_modules folder is currently copied into processes. The amount of IO is significant, with npm and yarn suffering the worst. Describe the solution you'd like Instead of merging installation digests with the sources to be acted upon, pass node_modules as immutable inputs. Describe alternatives you've considered ? Additional context pantsbuild/pants
  • c

    cool-easter-32542

    05/19/2023, 4:32 PM
    #17712 Replace `immutable_input_digests` with include/excludelist from `input_digest` Issue created by thejcannon Describe the bug #17520 now supports symlink-by-hueristic, we no longer need 
    input_digest
    as a process input. Instead this can be translated into being include-(and while we're at it, exclude)-listed. Pants version 
    main
    N/A pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/19/2023, 4:33 PM
    #17282 Include/exclude and heuristic-based `immutable_inputs` Issue created by stuhood 
    Process.immutable_inputs
    is currently configured as a mapping from a file path to a 
    Digest
    to materialize at that path. It is merged into the 
    input_digest
    of a 
    Process
    to form a "complete" 
    Digest
    of inputs. But this separation is a bit unfortunate, because it means that using 
    immutable_inputs
    (which should be fairly widely used: see discussion on #14070) means a lot of change to how your 
    Process
    is constructed. * * * From an API-changes perspective, we should: 1. remove the guarantee that inputs are mutable by default, to allow us to automatically choose the strategy that we use for each path, and symlink wherever we choose to. • Then, by default, we should automatically use symlinking for files or directories over a certain size, etc. 2. convert the existing 
    immutable_inputs
    list into a flat list of paths which should be _forced_/includelisted to be materialized as immutable (regardless of the heuristic from above), but which must already exist in the 
    input_digest
    (maybe "force" should go in the name? 
    force_immutable_paths=
    ...?). • This will mean that unlike today (where you need to change how your 
    Process
    is constructed to stop adding an input to the 
    input_digests
    and instead add it to the 
    immutable_inputs
    as a dict), you would only need to add a path to the 
    immutable_inputs
    , without changing your 
    input_digest
    . 3. add an 
    mutable_inputs
    (or 
    force_mutable_paths=
    ...?) flat list of paths to act as an excludelist that prevents a path from being materialized as a symlink. * * * From an implementation perspective, this would look like moving the support for creating the symlinks for 
    ImmutableInputs
    from an explicit step before running a process: pants/src/rust/engine/process_execution/src/local.rs Lines 641 to 662 in </pantsbuild/pants/commit/08410e5b94d3e3aa86b2538ee1991d964e27e72c|08410e5> ...to a thing that happens inside 
    fn materialize_directory
    when an includelisted path is encountered: pants/src/rust/engine/fs/store/src/lib.rs Lines 1188 to 1201 in </pantsbuild/pants/commit/08410e5b94d3e3aa86b2538ee1991d964e27e72c|08410e5> 
    materialize_directory(_helper)
    would take either: 1. a reference to a collection of paths which were includelisted as immutable 2. a trait or function which implemented the heuristic for which `directory::Entry`s / paths to includelist ...and a reference to 
    ImmutableInputs
    , and then would create a symlink to the 
    ImmutableInputs
    whenever it encountered one of the includelisted paths. pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/19/2023, 5:58 PM
    #19060 High memory usage spinning up Pants on clean Go monorepo Issue created by ImprintNav Describe the bug Upon performing a compile-check against a clean checkout of our Go repo with roughly 1000000 lines of Go code and 7690 Go files, the Pants daemon ends up consuming a peak of roughly 15 GiB of memory before cooling off. The operation also takes much longer than 
    go build ./...
    . Pants version 2.17.0.dev5 OS MacOS ARM64 Additional info When running 
    -ldebug
    , I notice that Go modules keep getting refetched over and over again. I don't think the daemon is actually dying this time around so it's unclear why this is happening. Result of 
    pants check --stats-memory-summary ::
    (omitted ldebug due to size and privacy for now) ``` 133150.47 [INFO] Memory summary (total size in bytes, count, name): 48 1 pants.backend.go.dependency_inference.AllGoModuleImportPathsMappings 48 1 pants.backend.go.dependency_inference.GoModuleImportPathsMapping 48 1 pants.backend.go.dependency_inference.GoModuleImportPathsMappings 48 1 pants.backend.go.goals.check.GoCheckRequest 48 1 pants.backend.go.subsystems.golang.GolangSubsystem 48 1 pants.backend.go.subsystems.golang.GolangSubsystem.EnvironmentAware 48 1 pants.backend.go.subsystems.gotest.GoTestSubsystem 48 1 pants.backend.go.target_type_rules.FirstPartyGoModuleImportPathsMappingsHook 48 1 pants.backend.go.target_type_rules.GenerateTargetsFromGoModRequest 48 1 pants.backend.go.target_type_rules.GoImportPathMappingRequest 48 1 pants.backend.go.util_rules.build_pkg.CheckForGolangAssemblyRequest 48 1 pants.backend.go.util_rules.build_pkg.CheckForGolangAssemblyResult 48 1 pants.backend.go.util_rules.build_pkg.SetupAsmCheckBinary 48 1 pants.backend.go.util_rules.build_pkg_target._ResolveStdlibEmbedConfigRequest 48 1 pants.backend.go.util_rules.build_pkg_target._ResolveStdlibEmbedConfigResult 48 1 pants.backend.go.util_rules.cgo.CGoCompilerWrapperScript 48 1 pants.backend.go.util_rules.cgo_binaries.CGoBinaryPathRequest 48 1 pants.backend.go.util_rules.go_bootstrap.GoBootstrap 48 1 pants.backend.go.util_rules.goroot.GoRoot 48 1 pants.backend.go.util_rules.implicit_linker_deps.SdkImplicitLinkerDependenciesHook 48 1 pants.backend.go.util_rules.link.LinkerSetup 48 1 pants.backend.go.util_rules.link_defs.ImplicitLinkerDependencies 48 1 pants.backend.go.util_rules.pkg_analyzer.PackageAnalyzerSetup 48 1 pants.backend.go.util_rules.sdk.GoSdkRunSetup 48 1 pants.backend.go.util_rules.third_party_pkg.AllThirdPartyPackages 48 1 pants.backend.go.util_rules.third_party_pkg.AllThirdPartyPackagesRequest 48 1 pants.backend.go.util_rules.third_party_pkg.ModuleDescriptors 48 1 pants.backend.go.util_rules.third_party_pkg.ModuleDescriptorsRequest 48 1 pants.backend.project_info.filter_targets.FilterSubsystem 48 1 pants.backend.python.goals.lockfile.PythonSyntheticLockfileTargetsRequest 48 1 pants.backend.python.subsystems.setup.PythonSetup 48 1 pants.base.build_root.BuildRoot 48 1 pants.build_graph.build_configuration.BuildConfiguration 48 1 pants.core.goals.check.Check 48 1 pants.core.goals.check.CheckResults 48 1 pants.core.goals.check.CheckSubsystem 48 1 pants.core.util_rules.distdir.DistDir 48 1 pants.core.util_rules.environments.AllEnvironmentTargets 48 1 pants.core.util_rules.environments.DockerPlatformFieldDefaultFactoryRequest 48 1 pants.core.util_rules.environments.EnvironmentNameRequest 48 1 pants.core.util_rules.environments.EnvironmentTarget 48 1 pants.core.util_rules.environments.EnvironmentsSubsystem 48 1 pants.core.util_rules.search_paths.ValidateSearchPathsRequest 48 1 pants.core.util_rules.search_paths.ValidatedSearchPaths 48 1 pants.core.util_rules.system_binaries.BashBinary 48 1 pants.core.util_rules.system_binaries.BinaryPath 48 1 pants.engine.console.Console 48 1 pants.engine.env_vars.PathEnvironmentVariable 48 1 pants.engine.environment.ChosenLocalEnvironmentName 48 1 pants.engine.fs.Workspace 48 1 pants.engine.internals.build_files.BuildFileOptions 48 1 pants.engine.internals.dep_rules.MaybeBuildFileDependencyRulesImplementation 48 1 pants.engine.internals.graph.ResolveTargetGeneratorRequests 48 1 pants.engine.internals.graph.ResolvedTargetGeneratorRequests 48 1 pants.engine.internals.graph.SubprojectRoots 48 1 pants.engine.internals.mapper.SpecsFilter 48 1 pants.engine.internals.options_parsing._Options 48 1 pants.engine.internals.parser.BuildFilePreludeSymbols 48 1 pants.engine.internals.parser.Parser 48 1 pants.engine.internals.synthetic_targets.AllSyntheticAddressMaps 48 1 pants.engine.platform.Platform 48 1 pants.engine.target.FieldDefaultFactoryResult 48 1 pants.engine.target.FieldDefaults 48 1 pants.engine.target.GeneratedTargets 48 1 pants.engine.target.RegisteredTargetTypes 48 1 pants.engine.target.TargetTypesToGenerateTargetsRequests 48 1 pants.engine.unions.UnionMembership 48 1 pants.goal.anonymous_telemetry.AnonymousTelemetry 48 1 pants.goal.anonymous_telemetry.AnonymousTelemetryCallbackFactoryRequest 48 1 pants.goal.stats_aggregator.StatsAggregatorCallbackFactoryRequest 48 1 pants.goal.stats_aggregator.StatsAggregatorSubsystem 48 1 pants.init.bootstrap_scheduler.BootstrapStatus 48 1 pants.option.global_options.GlobalOptions 48 1 pants.option.global_options.KeepSandboxes 48 1 pants.option.global_options.UnmatchedBuildFileGlobs 48 2 pants.engine.env_vars.CompleteEnvironmentVars 48 5 pants.base.specs.Specs 48 30746 pants.engine.environment.EnvironmentName 56 1 pants.backend.go.util_rules.go_mod.AllGoModTargets 64 1 pants.engine.streaming_workunit_handler.WorkunitsCallbackFactories 88 1 builtins.ProcessExecutionEnvironment 96 2 pants.backend.go.go_sources.load_go_binary.LoadedGoBinary 96 2 pants.backend.go.go_sources.load_go_binary.LoadedGoBinaryRequest 96 2 pants.backend.go.util_rules.build_pkg.RenderEmbedConfigRequest 96 2 pants.backend.go.util_rules.build_pkg.RenderedEmbedConfig 96 2 pants.backend.go.util_rules.cgo.CGoCompileRequest 96 2 pants.backend.go.util_rules.cgo.CGoCompileResult 96 2 pants.backend.go.util_rules.cgo.SetupCompilerCmdRequest 96 2 pants.backend.go.util_rules.cgo.SetupCompilerCmdResult 96 2 pants.backend.go.util_rules.go_mod.GoModInfo 96 2 pants.backend.go.util_rules.go_mod.GoModInfoRequest 96 2 pants.backend.go.util_rules.import_analysis.GoStdLibPackages 96 2 pants.backend.go.util_rules.import_analysis.GoStdLibPackagesRequest 96 2 pants.backend.go.util_rules.link.LinkGoBinaryRequest 96 2 pants.backend.go.util_rules.link.LinkedGoBinary 96 2 pants.engine.internals.graph.Owners 96 2 pants.engine.internals.graph.OwnersRequest 96 2 pants.engine.streaming_workunit_handler.WorkunitsCallbackFactory 144 1 (native) session_values 144 3 pants.backend.go.util_rules.build_pkg.BuiltGoPackage 144 3 pants.backend.go.util_rules.cgo.CheckCompilerSupportsFlagRequest 144 3 pants.backend.go.util_rules.cgo.CheckCompilerSupportsOptionResult 144 3 pants.backend.go.util_rules.sdk.GoSdkToolIDRequest 144 3 pants.backend.go.util_rules.sdk.GoSdkToolIDResult 144 3 pants.base.specs.RawSpecsWithOnlyFileOwners 144 3 pants.core.util_rules.system_binaries.BinaryPathRequest 144 3 pants.core.util_rules.system_binaries.BinaryPaths 144 3 pants.engine.env_vars.EnvironmentVars 144 3 pants.engine.env_vars.EnvironmentVarsRequest 273 1 (native) construct_scope_ 273 1 (native) construct_scope_anonymous_telemetry 273 1 (native) construct_scope_check 273 1 (native) construct_scope_environments_preview 273 1 (native) construct_scope_filter 273 1 (native) construct_scope_go_test 273 1 (native) construct_scope_golang 273 1 (native) construct_scope_python 273 1 (native) construct_scope_stats 273 1 (native) pants.backend.go.util_rules.cg… pantsbuild/pants
    • 1
    • 2
  • c

    cool-easter-32542

    05/20/2023, 10:43 AM
    #19068 javascript: Enable passing env vars to `node_build_script` Issue created by tobni Is your feature request related to a problem? Please describe. Quote from slack:
    It would be great if we could use extra_env_vars to set up environment variables for node_build_script. Many projects need compile time env vars to create a bundle.
    Describe the solution you'd like An 
    extra_env_vars
    field on 
    node_build_script
    . Describe alternatives you've considered Additional context pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/20/2023, 11:14 PM
    #17978 `[python] interpreter_constraints` is not used as the default IC for user resolves. Issue created by jsirois As of Pants 2.16.0.dev4 (main HEAD currently), the following configuration is needed to ensure the 
    python-default
    resolve gets the repo-local default IC and not the Pants-embedded default IC: pants/pants.toml Lines 130 to 147 in </pantsbuild/pants/commit/05b9465b70064d4d5be2ce02f011af1c73428ed9|05b9465> This feels like a bug. If the default ICs written down by the user are not picked as the default IC for a resolve written down by a user, what is the default IC used for then? pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/21/2023, 8:06 AM
    #6597 Update remote execution stubs to return status protos Issue created by illicitonion Right now, grpcio doesn't support servers sending protos in the status details field ("grpc-status-details-bin" trailing header). There is an ignored test which tests the behaviour of being sent a status proto. We should un-ignore it when we have a grpc stack which supports this. pantsbuild/pants
    • 1
    • 1
  • c

    cool-easter-32542

    05/21/2023, 8:12 AM
    #12552 Set default invalid lockfile behaviour to `error` Issue created by chrisjrn After setting the default behaviour to 
    error
    during development work for #12448, our CI stopped passing. I've since changed the behaviour back to 
    warn
    in order to land the PR. To address, we should clear out all invalid lockfile warnings from our CI runs, and once we've achieved that, change the default behaviour to 
    error
    . This can be addressed after #12542 is complete. pantsbuild/pants
    • 1
    • 1
1...282930...34Latest