Huh...

Thanks for the tip @clever-hamburger-59716

@happy-kitchen-89482 one more question-- I can debug my service pex with ipdb

ipdb.set_trace()

but normally (outside of pants and this project entirely) i have

PYTHONBREAKPOINT=ipdb.set_trace

in my envrionment and

breakpoint()

automatically gives me ipdb instead of pdb. this isn’t working with my pex and I’m stumped. At the point where I run

ipdb.set_trace()

I print

os.environ["PYTHONBREAKPOINT"]

and it’s

ipdb.set_trace

. let me see if I can make a tiny repro repo

if i set the variable to something bogus like

afs

I don’t get an error, while normally I do

so i am back to the problem in the original post. my pex depends on ipdb, but when I add it to requiremens.txt, I get this error, even though ipdb is definitely on https://pypi.org/simple/ipdb/ with a version suitable for python 3.10

stderr:
pid 75041 -> /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/07b01ddf09672a6b76d4e76683b2604821a7fda1/bin/python -sE /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/07b01ddf09672a6b76d4e76683b2604821a7fda1/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/ts/7lr9hnf958b68z7qr4dvtqvh0000gn/T/pants-sandbox-nBWpPv/.tmp/pex-pip-log.9lc3m4zm/pip.log download --dest /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/downloads/resolver_download.w2d38tzk/cp39-cp39-manylinux2014_x86_64 --only-binary :all: fsspec[http] ipdb lz4==4.0.2 modin@ git+<https://github.com/devin-petersohn/modin.git@service/init-prod> rpyc==5.2.3 segment-analytics-python==2.2.1 snowflake-connector-python[pandas]==2.7.11 --platform manylinux2014_x86_64 --implementation cp --python-version 39 --abi cp39 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement ipdb
ERROR: No matching distribution found for ipdb

ok i found a fix by comparing to a tiny repo that didn’t have the failure. but I don’t understand why it works.

pex_binary(
    name="service",
    entry_point="service.py",
    platforms=[
        "current",
        "manylinux2014-x86_64-cp-39-cp39",
    ],
    dependencies=["//:soda#modin", "ponder/redacted:redacted"],
)

when I comment out the platforms, I can get ipdb. when I add them, I can’t find it. @happy-kitchen-89482 what could be the problem?

@clever-hamburger-59716 is this linux platform spec too specific? i doubt every package has a version like that

Those platforms will cause Pants to build a pex compatible with both platforms.

ipdb

is only available as an sdist on PyPI, so it can only be built for

manylinux2014-x86_64-cp-39-cp39

on that platform

Thanks @happy-kitchen-89482 , we didn’t need manylinux2014, we removed that requirement.

Actually, looking back on it, should I even be relying on native tooling for features such as hot-reload when using Pants? For example when I build a fastapi server and run it with uvicorn, should I let uvicorn reload with a

--reload

flag or should I start the process with pants and let pants restart the process with a

--watch

or something else?

Is it expected that changing pants versions invalidates the local cache? To validate our future upgrade from 2.14.0, I ran both 2.15.0rc0 and 2.16.0.dev2 through our CI, without other code changes, against a bulk-cached

~/.cache/pants/lmdb_store

(not remote cache), and both of them seemed to run from scratch (metrics like

local_cache_requests_cached: 0

). Running locally seemed to be similar, too, but I didn't pay that much attention. (Both upgrades seemed to work fine in terms of delivering a working deploy, at least ✅ )

Can you prevent

--changed-since

for

./pants

to read changes at the root directory? (This was possible before with the

--changed-parent

option, but it has been deprecated: https://www.pantsbuild.org/v2.0/docs/reference-changed#deprecated-options) I am using

./pants 2.14.0

on a monorepo with multiple programming languages. pants is installed in the

py/mono

directory. It seems that if the root directory contain some changes, then running

./pants --changed-since=master package

throws an error similar to this:

Exception caught: (builtins.ValueError)
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/bin/pants", line 8, in <module>
    sys.exit(main())
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 115, in main
    PantsLoader.main()
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 111, in main
    cls.run_default_entrypoint()
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 93, in run_default_entrypoint
    exit_code = runner.run(start_time)
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/pants_runner.py", line 99, in run
    runner = LocalPantsRunner.create(
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 159, in create
    specs = calculate_specs(
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/init/specs_calculator.py", line 90, in calculate_specs
    changed_files = tuple(changed_options.changed_files(maybe_git_worktree.git_worktree))
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/vcs/changed.py", line 125, in changed_files
    git_worktree.changed_files(
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/vcs/git.py", line 107, in changed_files
    return {self._fix_git_relative_path(f, relative_to) for f in files}
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/vcs/git.py", line 107, in <setcomp>
    return {self._fix_git_relative_path(f, relative_to) for f in files}
  File "/home/circleci/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages/pants/vcs/git.py", line 70, in _fix_git_relative_path
    return str((self.worktree / worktree_path).relative_to(relative_to))
  File "/home/circleci/.pyenv/versions/3.9.15/lib/python3.9/pathlib.py", line 939, in relative_to
    raise ValueError("{!r} is not in the subpath of {!r}"

Exception message: '/home/circleci/br-monorepo/dev.tmpl' is not in the subpath of '/home/circleci/br-monorepo/py/mono' OR one path is relative and the other is absolute.
NoneType: None

Looking at the pants source code, it seems

./pants

will use the git API to obtain all the changed files, and then compute the path relative to the build root:

git_worktree.changed_files(                                                             
   from_commit=changes_since, include_untracked=True, relative_to=get_buildroot()      
),

For example, if my build root is py/mono and a file is found at py/mono/a/b/c, then it will return a/b/c So it seems that it will throw an error if the changed file is outside the pants directory. Is it possible to prevent this behavior, i.e., just ignore those files?

Is there a way to clear the pants cache? I've been trying to figure out an issue with my project structure, Finally just deleted the cache and re-ran tailor, which fixed the issue. I was expecting that to happen automatically, but I guess it wouldn't if I didn't change any files? It's really weird, now if I edit

pants.toml

for example, when I run pants again it doesn't seem to know that the edit has been made. I get the same error message that caused me to update. But if I delete

.pants.d

and

pids

then run it works.

I'm seeing

pex

have issues resolving dependencies and report conflicts when both

pip

and

pipenv

work fine. I've narrowed down the issue to a set containing just these two dependencies:

"grpcio<1.48.1" "google-api-core[grpc] >= 1.22.1"

I suspect this has to do with how the extras

[grpc]

is handled. Details in thread.

I think this has been asked already but I'm struggling to run random script wrapped inside a PEX which is generated using

pex_binary

in

venv

mode. I didn't set any of these scripts as an entry point. Is this possible?

Hello everyone, I trying Pants since last Friday and everything is looking smooth so far (trying to migrate ~30 microservices to a monorepo structure with pants). One thing I cant solve if the

flake8

plugins, I have define a

.ini

file and define in my

pants.toml

my file

[...]

[flake8]
config = "thirdparty/flake8/flake8.ini"
version = "flake8==6.0.0"
lockfile = "thirdparty/flake8/.lock"

My main problem is the exclude instruction of the

.ini

file

[flake8]
max-line-length = 120
max-complexity = 15
extend-ignore =
  RST902
exclude =
    migrations,
    docs,
    dist,
    build,
    .eggs,
    docker,
    venv,
    sphinx,
    .pytest_cache,
    .tox
    versioneer.py

the exclude is working since a lots of those folder are parsed correctly and does not appear in the

./pants lint ::

command, but the

migrations,

won't work, I have tried so many patterns to make is works,

*migrations,*migrations*,migrations/

also tried to give the absolute path (just to test) and I still having all my migrations folder of all my projects discovered by flake8 (the exclude list is working if running from within the project with the same config file. I have cleaned the cache between every tests 😞 with no luck, any guess on this ? please

I was just reading in the GitHub issues that everything here is being lost after 90 days: https://github.com/pantsbuild/pants/issues/16534#issuecomment-1218140696 Other very successful open source python communities are build built in Discourse: https://discuss.streamlit.io/ And, if you ask them about it they will often provide a completely free hosted solution for open source communities.

I have a weird issue where hadolint fails when pants run it but works if I run it manually by directly calling

hadolint

I don’t even get very helpful error message from pants. No idea what

exit code -11

supposed to mean.

12:48:24.11 [ERROR] Completed: Lint with Hadolint - hadolint failed (exit code -11).

Does anyone know if there's been any recent progress in relation to using Pants with MyPy's daemon?

Hi, I had been on 1.1.14 version of poetry and the requirements.txt didn't contain any optional or extras. Today I upgraded poetry and now requirements.txt contains extras and I am seeing the test cases run by pants are failing. Here's my pyproject.toml file -> from which requirements.txt is generated via

poetry export -f requirements.txt -o requirements.txt --without-hashes

python_requirement(
    name="setuptools",
    requirements=["setuptools"],
)

python_requirement(
    name="torch",
    requirements=["torch==1.10.0+cpu"],
    dependencies=[":setuptools"],
)

python_requirement(
    name="pytorch-lightning",
    requirements=["pytorch-lightning"],
    dependencies=[":torch"],
)

python_requirement(
    name="torchmetrics",
    requirements=["torchmetrics"],
    dependencies=[":torch"],
)

python_requirements(
    name="reqs",
    module_mapping={
        "python-constraint": ["constraint"],
        "google-api-core": ["google.api_core"],
        "grpcio": ["grpc"],
        "grpcio-health-checking": ["grpc_health"],
        "python-json-logger": ["pythonjsonlogger"],
    },
)

It looks like pants treats requirements.txt as a constraints file(

ERROR: Constraints cannot have extras

). Do I need to upgrade pants or make some changes in BUILD file to resolve the issue?

Hi team, I am facing the following issue while using the pants 2.14 binary published here in Apple M1 book. It seems to me the binary does not support M1 book? Am I right?

./pants --version
Failed to find compatible interpreter on path /Users/266409531:/Users/266409531/Dev/apache-maven-3.8.5/bin:/opt/miniconda3/envs/brain_toy/bin:/opt/miniconda3/condabin:/Users/266409531:/Users/266409531/Dev/apache-maven-3.8.5/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin.

Examined the following interpreters:
1.)                                                        /opt/miniconda3/envs/brain_toy/bin/python3.9 CPython==3.9.13
2.) /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8 CPython==3.8.9

No interpreter compatible with the requested constraints was found:

  A distribution for pantsbuild-pants could not be resolved for /opt/miniconda3/envs/brain_toy/bin/python3.9.
  Found 6 distributions for pantsbuild-pants that do not apply:
  1.) The wheel tags for pantsbuild.pants 2.14.0+gitfae1f69f are cp37-cp37m-macosx_10_16_x86_64 which do not match the supported tags of /opt/miniconda3/envs/brain_toy/bin/python3.9:
  cp39-cp39-macosx_12_0_arm64
  ... 367 more ...
  2.) The wheel tags for pantsbuild.pants 2.14.0+gitfae1f69f are cp37-cp37m-manylinux2014_x86_64 which do not match the supported tags of /opt/miniconda3/envs/brain_toy/bin/python3.9:
  cp39-cp39-macosx_12_0_arm64
  ... 367 more ...
  3.) The wheel tags for pantsbuild.pants 2.14.0+gitfae1f69f are cp38-cp38-macosx_11_0_x86_64 which do not match the supported tags of /opt/miniconda3/envs/brain_toy/bin/python3.9:
  cp39-cp39-macosx_12_0_arm64
  ... 367 more ...
  4.) The wheel tags for pantsbuild.pants 2.14.0+gitfae1f69f are cp38-cp38-manylinux2014_x86_64 which do not match the supported tags of /opt/miniconda3/envs/brain_toy/bin/python3.9:
  cp39-cp39-macosx_12_0_arm64
  ... 367 more ...
  5.) The wheel tags for pantsbuild.pants 2.14.0+gitfae1f69f are cp39-cp39-macosx_10_16_x86_64 which do not match the supported tags of /opt/miniconda3/envs/brain_toy/bin/python3.9:
  cp39-cp39-macosx_12_0_arm64
  ... 367 more ...
  6.) The wheel tags for pantsbuild.pants 2.14.0+gitfae1f69f are cp39-cp39-manylinux2014_x86_64 which do not match the supported tags of /opt/miniconda3/envs/brain_toy/bin/python3.9:
  cp39-cp39-macosx_12_0_arm64
  ... 367 more ...

I have an application that has some files that need to follow, so I packaged them all in an 'archive'. The files are deployed in the same location as the pex binary. This works fine, and I can also tweak it to be able to run from 'dist/app.pex' by providing tha data directory as a parameter. But it doesn't work well with './pants run', because the current directory is changed to the tmp one and the data dir has to be an absolute path. This makes it a bit cumbersome to debug. Is there a simpler way that I missed?

is there any way to override pinned dependencies of a third party pypi packages? There are many packages out there that just assumes we will have a small environment and pins all the dependencies. Rather than forking them or having to convince the author to relax the pinned version, it will be nice if we can have some control over them on our end

I’m trying to package a

python_distribution

that depends on some

protobuf_sources

in a sibling directory. What do I need to specify in my dependencies field in order to satisfy the ownership constraint? Right now I have the following:

python_sources(name="lib")

python_distribution(
    name="auth_service_client",
    dependencies=[
        ":lib",
        "src/proto/auth_service/service.proto:auth_service_interfaces",
    ],
    provides=python_artifact(
        name="auth_service_client",
        version="0.0.1",
    ),
)

This is in the subdirectory that has my python sources for the library. It depends on some codegen protocol buffers in a sibling directory with the following BUILD file:

protobuf_sources(
    name="auth_service_interfaces",
    python_source_root="src/python",
    grpc=True,    
)

Pants is aware of the dependency between the

python_source

generated by `python_sources`:

./pants dependencies src/python/auth_service_client/client.py
src/proto/auth_service/service.proto:auth_service_interfaces
src/python:reqs#grpcio

But when I try to package, I get the following error message:

NoOwnerError: No python_distribution target found to own src/proto/auth_service/service.proto:auth_service_interfaces. Note that the owner must be in or above the owned target's directory, and must depend on it (directly or indirectly). See <https://www.pantsbuild.org/v2.14/docs/python-distributions> for how python_sources targets are mapped to distributions. See <https://www.pantsbuild.org/v2.14/docs/python-distributions>.

I read through the docs but I’m not understanding how to map the

protobuf_sources

to a

python_sources

target that my distribution can depend on. What am I missing here?

Did you propose a talk or tutorial to Pycon US? If it's related to building software or other topics relevant to our community, I'd love to know. Tell us what you're up to here!

Has anyone used pants to have a single mega repo of “I do all my dev in this”, regardless of if it’s one project or not?

Can relocated_files be used with resources in a configurable way? If not, is there a way to add resources to a pex to a specific place without physically locating them in the source directory? I need to have them available as a separate directory, and in the pex they should have some directories trimmed. Right now I get them with the same directory structure as in the repository.

Will any of the future releases allow for utilizing the more incremental form of computing a lock file? I believe pex supports not recomputing from scratch each time.

If I wanted every python target under a certain folder to use a specific resolve instead of the

default_resolve

, would I need to use a macro for that to avoid typing

resolve=<my resolve>

everywhere?

Any recommended best practices for having multiple resolves while internally sharing first-party modules/resources. It seems like if you had a project tree with a more general library, you'd have to go update what resolves that general library is discoverable under each time a new project with a different resolve is added. Is that correct?