I’ve got a predicament with our local mac, and linux based CI/CD processes. We have a PEX artifact that gets packaged into a docker container. Due to some wheel compatibility issues, the PEX must be build for a linux x86 system. To get around this issue we started using environments (in particular a

docker_environment

) to build the PEX and subsequent

docker_image

target. Everything works great locally until it came time to publish our

docker_image

from CI/CD (GitHub Actions self-hosted runners). The build fails with the following error when I specify a

docker_environment

on the

pex_binary

target:

Exception: Failed to obtain version from local Docker: error trying to connect: No such file or directory (os error 2)

As best as I can tell the

DOCKER_HOST

environment variable isn’t getting passed down into the build process and it’s not finding the docker binary when trying to run

docker -v

. I’ve specified

DOCKER_HOST

and a number of other vars in the

docker.env_vars

table and have also tried supplying them to the

docker_environment

target itself. I’ve seen a few somewhat-related issues around this in the slack history, and so I’m wondering if there’s an issue with my pants config or if

docker_environment

just isn’t ready for primetime. (I’m on pants 2.15.0, but have tried other versions too)

Asking for advice about how to handle pytest coverage especially with the

--changed-since

flag. We used to enforce coverage minimums because we could run tests per module in a monorepo, and determine the coverage percentage by that module. However, if multiple files have changed, and we rely on the --changed-since flag to run whatever tests we think we need to run, I’m not sure how to get a reliable measure of coverage. The only way I think we can do this is to run all tests and rely on caching for speed, along with

--coverage-py-global-report

Any other ideas?

Hoping someone can help me out here, I started getting an error today when launching pants (v2.14):

raise BackendConfigurationError(f"Failed to load the {backend_module} backend: {ex!r}")
pants.base.exceptions.BackendConfigurationError: Failed to load the pants.core.register backend: ImportError("urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips  26 Jan 2017. See: <https://github.com/urllib3/urllib3/issues/2168>")

Seems that a new version of the

requests

library was released this morning which added support for urllib3 v2.0. Wondering if there is a way that I can pin an older version to fix this.

Using

2.0.17.dev4

I’m starting to see failures like

Engine traceback:
  in `package` goal

InvalidTargetNameError: The target name ../tests (defined in directory iai_coral_api/flg_tests/agents/test_rest.py) contains banned characters (`/`). Please replace these characters with another separator character like `_` or `-`.

The test BUILD files do a recursive glob to register the test names. Seems related to that. I can sometimes get around this by specifying more BUILD files in subdirectories, but it makes BUILD management onerous.

📣 Reminder: we've got a Pants Jobs page! If you're looking for engineers who are passionate about build engineering, code quality, developer experience, and scaling, then this is the community you want to recruit in. Feel free to contribute job postings at any time. The jobs page has helped get a number of people into terrific new jobs where their Pants skills are valued—including several members of core team. It'd be a pleasure to help you connect to a talented engineer who shares your team's values. Cheers!

Probably a silly question:

Following the deprecation cycle in 2.15, all Goals need to set EnvironmentBehavior.LOCAL_ONLY or EnvironmentBehavior.USES_ENVIRONMENTS.

Where do we set this for the Goal?

Can I use different set of custom PIP indexes for different resolves? Pants/Pex doesn't generate a portable lockfile if we include an index in our primary lockfile; but we need it for other more niche applications and workflows.

Hey, I just created an issue but maybe anyone here can help me: https://github.com/pantsbuild/pants/issues/18896

Does anyone know if the

pants run

goal supports being run inside a docker_environment, and if so, how to set this up?

Greetings Pants friends! I have a script that produces our Flutter build artifacts called, creatively,

./build.sh

. I am able to configure this to run successfully using Pants 1.16

run_shell_command

. Executing via

./pants run :build-flutter

produces the expected artifacts. These are then copied into our docker container. If the files are present in the source files, then the

files

and

docker_image

targets work perfectly. I'm having trouble sequencing dependencies so that the script is executed before other targets. Is this supported? Here's my BUILD file

files(
    name="flutter-source",
    sources=[
        "./**",
    ]
)

shell_sources(name="shell-scripts")

# produces build artifacts in source code workspace
run_shell_command(
    name="build-flutter",
    command="cd realtime/cloud_ui && ./build.sh",
    execution_dependencies=[":shell-scripts", ":flutter-source"]
)

files(
    name="docker-assets",
    dependencies=[":build-flutter"],
    sources=[
        "./Caddyfile",
        "./docker_entrypoint.sh",
        "./build/web/**",
    ]
)

docker_image(
    name="neuroedge_cloud_ui",
    dependencies=[
        ":docker-assets"
    ],
    image_tags=["latest"]
)

when I execute

./pants package :neuroedge_cloud_ui

, I expect the dependency chain will invoke

:build-flutter

to produce the required "_./build/web/**"_ files matched by

:docker-assets

but this is not the case. If I run

:build-flutter

and then package

:neuroedge_cloud_ui

all is well. Is it possible to ensure that

:build-flutter

is always run as a dependency of

:build-flutter

?

Is there a sensible way to change the constraints for pants? Currently, it will pull

requests

2.30

(even though it's yanked, but I guess there'll be a fix out soon) which pulls

urllib3

2.0.2

, which isn't compatible with

botocore

. We can probably play whackamole with

GLOBAL.plugins

, but we also need to run pants on older machines (with an

openssl

older than

1.1.1

, which `urllib3 requires), and then it'd fail to install pants, before resolving the plugins.

We're going through a very painful exercise of upgrading our apps to python 3.10. As part of this, we go through libraries and we mark them as compatible by adding

interpreter_constraints=[
        "CPython~=3.7.4",
        "CPython~=3.10.9",
    ]

to

python_sources

and

interpreter_constraints=parametrize(
        py37=["~=3.7.4"],
        py310=["~=3.10.9"],
    ),

to our

python_tests

. The idea being, we mark libs as compatible, then run tests on both. So this works sort of okay, until this interacts with init file inference (here). When this has the default value, we get:

pants.engine.target.InvalidFieldException: The target filters/src/messages/tests/unit/templates/test_helpers.py:../../../test@interpreter_constraints=py37 has the `interpreter_constraints` ('~=3.7.4',), which are not a subset of the `interpreter_constraints` of some of its dependencies:

  * ('~=3.10.9',): filters/src/messages/tests/unit/templates/__init__.py:../../../test@interpreter_constraints=py310

To fix this, you should likely adjust filters/src/messages/tests/unit/templates/test_helpers.py:../../../test@interpreter_constraints=py37's `interpreter_constraints` to match the narrowest range in the above list.

Aside turning that off, what can we do?

Hey all, we've got 2 projects in our monorepo that each have a config file with the same name. In our unit tests we relocate that config file to the sandbox directory using the

relocated_files

goal. However, the coverage plugin is failing now due to the following intrinsic error:

IntrinsicError: Can only merge Directories with no duplicates, but found 2 duplicate entries in :

This seems to be the problematic source code in question. Is this a legitimate bug or is there a way around this issue?

Is there anyone who can help us with this? It looks like

pants

is getting stuck on installing

[GLOBAL].plugins

, but it’s not consistently reproducible and I’m not sure how we can get more debugging output. At this point we’ve ripped out all plugins that use 3rd party dependencies because we could not figure out what was going on. Should I assume that 3rd party dependencies for plug-ins are unreliable?

Is there anyone with Azure WebApp service who has run PEXes (or preferably `scie`s/custom executables)? I'm kinda being force into using that for a deliverable - and it seems to dictate a specific configuration which is contrary to my whole setup (pex inside of a standalone fat scie). Has someone tried it running a PEX_TOOLS=1 pex venv before? I'll be trying it out this weekend, just wondering if I can get a heads up of the pain I'm in for

Is there a way to globally shut off python dependency inference? I thought there was one, but I only see it available https://www.pantsbuild.org/docs/reference-python-infer

I have some questions about working with multiple repos. My setup is • repo external-lib is the source for a Python package, external-lib; uses Pants • repo my-monorepo is my code, and the code here depends on external-lib; uses Pants if I want to locally develop external-lib alongside my-monorepo, how do I model that to Pants so it uses my local version of external-lib instead? Is there an equivalent to having an editable install aka defining a dependent package in development mode? End goal is to be able to run tests or scripts in my-monorepo with the local changes to external-lib. It seems a git submodule would probably get me what I want, but I was hoping to avoid that given the overhead of learning, teaching, and managing git submodules. Some options I’ve figured from the docs: 1. use python-repos.find_links to point to the local repo, and generate distributions in the external-lib repo, but that seems like a little bit of a pain to do each time I make a change and want to run tests or something 2. I did see that Pants has an option to export a venv so maybe the path, at least for running scripts, is to export a mutable venv, and do an editable install into there Are there other, better (more streamlined) options here?

Hello.. I am pretty new to Pants.. I was trying to use pants to generate docker images for a python application.. I am facing an issue when i run the tailor command..

✗ pants tailor ::
15:36:48.18 [ERROR] 1 Exception encountered:

  Exception: While expanding link "bin/python": Failed to read link "/Users/manitalanki/Work/biocloud_data_ingestion/bin/python3": Absolute symlink: "/Library/Developer/CommandLineTools/usr/bin/python3

what am I missing?

Running into an issue with pants in CI. Anyone seen this?

Engine traceback:
  in Update all BUILD files
  in Finding the `bash` binary
  in Scheduling: Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin

Exception: Failed to execute: Process {
    argv: [
        "./find_binary.sh",
        "bash",
    ],
    env: {
        "PATH": "/usr/bin:/bin:/usr/local/bin",
    },
...
    description: "Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin",
    ...
}

Error launching process: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }

Is there a way that I can set the timout for a remote cache write task? I’m getting this error whenever what I’m trying to save to the remote cache is on the larger side (in this case a node_modules folder):

16:18:46.03 [DEBUG] waiting for 6 session end task(s) to complete
16:18:49.04 [DEBUG] 1 session end task(s) failed to complete within timeout: remote cache write Digest { hash: Fingerprint<8c7d3c7d867f0731011568b072cb5d999f8e492dd946bdecee1549868d9eb9c3>, s
ize_bytes: 142 }

I tried setting

remote_cache_rpc_timeout_millis = 30000
remote_store_rpc_timeout_millis = 30000

Thinking that might have been the issue but it didn’t do anything. Any ideas?

I'm running into an error retrieving stubs when running

pants check ::

. I've reproduced the error in a minimal setup. Basically, I have one file which imports a third-party dep and a mypy_stub to include types for that 3rd party dep.

# ./modules/validator.py
from email_validator import EmailNotValidError, validate_email

# ./mypy_stubs/email_validator/__init__.pyi
... contains some typing info ...

pipenv run mypy

runs successfully on these files. However,

./pants check ::

will give me this error:

modules/validator.py:1: error: Skipping analyzing "email_validator": module is installed, but missing library stubs or py.typed marker

My setup also exists here in this repo: https://github.com/chrisplim/pants-check-error-mypy-stub

Reminder: Monday is Pants Team meetup. Team members, connection information is in your calendar invite. Folks, please message me ASAP with agenda items.

In my monorepo I use multiple resolves, mostly because apps are deployed to different environments and use different sources. How should I go with using one virtualenv for working in IDE (vscode)? Would I need to generate single requirements.txt file with all dependencies and then use virtualenv/pipenv/pdm or some other virtualenv or there is nicer way to do it?

Hi, with the following build i would expect the

version.txt

to be generated by the

vcs_version

and then able to use it in the

docker_image

docker_image(
    name="website-preview",
    source="Containerfile",
    dependencies=[":sources", ":version"],
    repository="docapse/website",
    image_tags=["pr-{build_args.PULL_REQUEST}"],
    description="Build website preview docker image",
    extra_build_args=["PULL_REQUEST"],
)

vcs_version(
    name="version",
    generate_to="website/version.txt",
    template='{version}',
)

But this does not seem to work, not sure what i am missing

Good afternoon I am running django project using pants but it seems not to import the indirect dependecies I have tried to use module mapping but I still get this error

Traceback (most recent call last):
  File "/home/kennan/.cache/pants/named_caches/pex_root/venvs/c327cad8531010afb442b743400c471a24792439/dee6656b00a07c8ae98b1c0dfcda4e61ad3e5014/pex", line 236, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/usr/lib/python3.9/runpy.py", line 225, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/kennan/NPA/fleet-backend/fleetdatabaseFinal/manage.py", line 21, in <module>
    main()
  File "/home/kennan/NPA/fleet-backend/fleetdatabaseFinal/manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/home/kennan/.cache/pants/named_caches/pex_root/venvs/c327cad8531010afb442b743400c471a24792439/dee6656b00a07c8ae98b1c0dfcda4e61ad3e5014/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/home/kennan/.cache/pants/named_caches/pex_root/venvs/c327cad8531010afb442b743400c471a24792439/dee6656b00a07c8ae98b1c0dfcda4e61ad3e5014/lib/python3.9/site-packages/django/core/management/__init__.py", line 420, in execute
    django.setup()
  File "/home/kennan/.cache/pants/named_caches/pex_root/venvs/c327cad8531010afb442b743400c471a24792439/dee6656b00a07c8ae98b1c0dfcda4e61ad3e5014/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/kennan/.cache/pants/named_caches/pex_root/venvs/c327cad8531010afb442b743400c471a24792439/dee6656b00a07c8ae98b1c0dfcda4e61ad3e5014/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/home/kennan/.cache/pants/named_caches/pex_root/venvs/c327cad8531010afb442b743400c471a24792439/dee6656b00a07c8ae98b1c0dfcda4e61ad3e5014/lib/python3.9/site-packages/django/apps/config.py", line 193, in create
    import_module(entry)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'rest_framework'

this is my build file at 3rdparty/build

python_requirement(
    name="djangorestframework",
    requirements=["djangorestframework>=3.12.2"],
    modules=["rest_framework"],
)
python_requirement(
    name="djangorestframework-simplejwt",
    requirements=["djangorestframework-simplejwt>=4.4.0"],
    modules=["rest_framework_simplejwt"],
)


python_requirements(
    name="reqs",
    module_mapping={"djangorestframework": ["rest_framework"],
                    "djangorestframework-simplejwt": ["rest_framework_simplejwt"]
                    }
                
)

This my requirements.txt files

flask==2.2.3
ansicolors>=1.0.2
Django==4.1.7
django-debug-toolbar>=3.8.0,<4
django-stubs>=1.10.0
gunicorn
setuptools>=42.0.0
translate>=3.2.1
protobuf>=3.11.3
pytest>=6.0.1
requests>=2.25.1
types-requests>=2.25.1
psycopg2-binary
python-dotenv>=0.20.0

I would appreciate all the assistance given

Is there an alternative of making dependencies that expose a different module name from their project name work because I get the same errror with the python-dotenv dependency

hey Pants! I have created a github issue for github-actions of init-pants https://github.com/pantsbuild/actions/issues/22

just to follow up on my earlier "fat sdist" question, would it be reasonable to add an option to generate fat sdist, as oppose to making reference to other sdists? If the code isn't too complicated to work on, I might have some spare time to take a look