Can pants work with different linting and formatting settings, specified individually in each project's

pyproject.toml

in a monorepo, or does it only take one config file total and only do config file discovery in the repo root?

I have been trying to install

scie-pants

in our CI/CD pipeline which runs inside a docker container on AWS. Even after modifying the PATH with

~/bin

, pants is not found. What am I missing? I run these commands

"export PATH=$PATH:~/bin"
"./get-pants.sh"
"pants --version"

This is the log from the container

[Container] 2023/03/20 11:28:34 Running command export PATH=$PATH:~/bin
[Container] 2023/03/20 11:28:34 Running command ./get-pants.sh
Downloading and installing the pants launcher ...
Installed the pants launcher from <https://github.com/pantsbuild/scie-pants/releases/latest/download/scie-pants-linux-x86_64> to /root/bin/pants
Running `pants` in a Pants-enabled repo will use the version of Pants configured for that repo.
In a repo not yet Pants-enabled, it will prompt you to set up Pants for that repo.
[Container] 2023/03/20 11:28:35 Running command pants --version
/codebuild/output/tmp/script.sh: 4: pants: not found

What is the network mode a container is started with when I use a

docker_environment

for my tests? I am trying to connect to something running on my localhost but I am getting a

ConnectionTimeout

. Am I able to change the network settings for my

docker_enviroments

?

How does one setup auth for a

docker_environment

? I have a private docker registry and thus setup

docker.env_vars

in pants.toml - using

docker_image

can pull from there just fine. When I try to use

docker_environment

though I get a 500 “no basic auth credentials”. I don’t see anything in the docs about this, or about how to point a

docker_environment

to a local

docker_image

as a workaround.

Is there a built-in way to just read

python_artifact

kwargs from

setup.cfg

or do I need to write a plugin for that?

So I have this step in our current build process that uses this tool to inject copyright headers before packaging. This is done in CI and hence the modified files are not in source control. Now, as part of the tarball and docker builds, it would be nice for the files to be modified before packaging (e.g. in the sandbox for example). Is there a good way/pattern to do this? Or do I have to check in all the code with copyright headers?

Scenario: I'm building a lambda function that depends on aws-lambda-powertools and boto3. Accordingly, I need to add an override to my python_dependencies to pull in jmespath and pydantic, which are undeclared transitive deps. When I deploy my lambda function, I want to elide those deps because they'll ship as part of a lambda layer. My lambda func is a pex_binary:

pex_binary (
    name="token_refresher",
    layout="zipapp",
    dependencies=[
        "myhandler.py",
        "!!3rdparty/python#boto3",
        "!!3rdparty/python#aws-lambda-powertools"
    ]
)

I'd have expected that jmespath and pydantic would be excluded, because they're transitive dependencies of two excluded deps, but by default they're pulled into the pex, so I have to exclude those as well. Is that intentional behaviour?

Hey, I'm getting this error when trying to run

pants check

after configuring a custom linter plugin.

12:11:57.96 [ERROR] 1 Exception encountered:

Engine traceback:
  in `check` goal

InvalidLockfileError: You are using the `pants-plugins` lockfile at 3rdparty/python/pants-plugins.lock with incompatible inputs.



- The inputs use interpreter constraints (`CPython==3.8.*`) that are not a subset of those used to generate the lockfile (`CPython==3.9.*`).- The input interpreter constraints are specified by your code, using the `[python].interpreter_constraints` option and the `interpreter_constraints` target field.

- To create a lockfile with new interpreter constraints, update the option `[python].resolves_to_interpreter_constraints`, and then generate the lockfile (see below).

See {doc_url('python-interpreter-compatibility')} for details.

To regenerate your lockfile, run `pants generate-lockfiles --resolve=pants-plugins`.

See <https://www.pantsbuild.org/v2.16/docs/python-third-party-dependencies> for details.

My default interpreter is Python 3.8, but I have my

pants-plugins

resolve set to use Python 3.9 in my

pants.toml

[python.resolves_to_interpreter_constraints]
pants-plugins = [ "CPython==3.9.*" ]

We're suddenly seeing

./pants package ::

take a lot longer than it used to do in CI. We have a case where it took 14 hours and crashed with

35d72d/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 223, in _main                                        
    status = self.run(options, args)                                                                                              
  File "/root/.cache/pants/named_caches/pex_root/venvs/18c001f81959497d649cc48967f7f8493f48767e/cba9b10390762edd539c75c9f4b40c5643
35d72d/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper                                       
    return func(self, options, args)                                                                                              
  File "/root/.cache/pants/named_caches/pex_root/venvs/18c001f81959497d649cc48967f7f8493f48767e/cba9b10390762edd539c75c9f4b40c5643
35d72d/lib/python3.10/site-packages/pip/_internal/commands/download.py", line 130, in run                                         
    requirement_set = resolver.resolve(                                                                                           
  File "/root/.cache/pants/named_caches/pex_root/venvs/18c001f81959497d649cc48967f7f8493f48767e/cba9b10390762edd539c75c9f4b40c5643
35d72d/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 121, in resolve                        
    self._result = resolver.resolve(                                                                                              
  File "/root/.cache/pants/named_caches/pex_root/venvs/18c001f81959497d649cc48967f7f8493f48767e/cba9b10390762edd539c75c9f4b40c5643
35d72d/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve                                    
    state = resolution.resolve(requirements, max_rounds=max_rounds)                                                               
  File "/root/.cache/pants/named_caches/pex_root/venvs/18c001f81959497d649cc48967f7f8493f48767e/cba9b10390762edd539c75c9f4b40c5643
35d72d/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 364, in resolve                                    
    raise ResolutionTooDeep(max_rounds)                                                                                           
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000

I have a project that builds a whl through

python_distribution

and I have tests that I want to run against this whl. I setup 2 resolves, one for building the package and one for running the tests. The tests resolve depends on the generated whl by setting

python-repos.find_links = ["file://%(buildroot)s/dist"]

, and this works if the whl has already been made. Pants doesn’t seem to infer that it should always package the whl before running the tests, and I don’t see a way to tell pants that it should other than making a custom target. I can do both steps manually in my CI, but that to me defeats the purpose of using pants. Is there an easy way to do this?

What targets does

black

target? Still struggling with the issue I posted about above. It's clearly picking up something it's not supposed to but I cannot determine what that is. If I run

pants --black-skip fmt lint ::

it works fine.

We have some Pydantic data models , that we need to code generate to

jsonschema

- (openapi) The "generation" is as simple as

from pydantic import Basemodel
...

class MyPydanticModel(BaseModel):
    ...

print(json.dumps(MyPydanticModel.schema(),indent=2))

I am happy plugging my way through writing a codegen plugin for this, but was wondering if I am able to leverage the openapi source types that exists already (for linting a openapi_document, etc) just after some guidance really. and once compete, I am happy to submit this plugin

Hi! How do I suppress or fix the warning:

The lint goal was called with targets including `<redacted>:bin-deps`, `<redacted>:bin-srcs`, `<redacted>:bin-deps` (and others), which specify the environment `docker_linux`, which is a `docker_environment`. The lint goal only runs in the local environment. You may experience unexpected behavior.

One more question: if I setup pants.toml with:

[docker]
env_vars = ["GITHUB_SHA=development"]
build_args = ["GITHUB_SHA"]

[environments-preview.names]
docker_linux = "//:docker_linux"
local_mac = "//:local_mac"
local_linux = "//:local_linux"

And BUILD file:

local_environment(
    name="local_mac",
    compatible_platforms=["macos_arm64"],
    fallback_environment="local_linux",
)

local_environment(
    name="local_linux",
    compatible_platforms=["linux_x86_64"],
    fallback_environment="docker_linux",
)

docker_environment(
    name="docker_linux",
    platform="linux_x86_64",
    image="python:3.11.2-slim-bullseye",
    python_bootstrap_search_path=["/usr/local/bin"],
)

I get an error when I run

pants package ::

:

[ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

DockerBuildContextError: Undefined value for build arg on the <redacted>:app target: The Docker environment variable 'GITHUB_SHA' is undefined. You may provide a value for this variable either in `[docker].env_vars` or in Pants's own environment.

If you did not intend to inherit the value for this build arg from the environment, provide a default value with the option `[docker].build_args` or in the `extra_build_args` field on the target definition. Alternatively, you may also provide a default value on the `ARG` instruction directly in the `Dockerfile`.

It looks like a bug

Hello all! I'm a Pants noob trying it as a way to get my multiple Django projects under control and have run in to an roadblock onboarding it. My goal is to slowly introduce the build system by initially putting all of my projects under a single

src

folder, and then slowly breaking out the shared modules/apps to get rid of all the duplication. I've moved the first repo under the

src

folder, set my

PYTHONPATH=src

env var, and in the

pants.toml

file I've set the source root as

root_patterns = ["/src"]

. However, this doesn't seem to be working. When I try to run the Django dev server using the command

pants run src/djangoproj1/manage.py -- runserver

I get the error

ModuleNotFoundError: No module named 'djangoproj1.settings'

. Fully aware I could be holding it wrong and missed a configuration setting while following the docs, but I would appreciate any help. I've setup a minimum reproduction using the

example-django

repo from the Pants org trying to best emulate what I did in my private repo, here's the relevant commit with all the changes and a comment containing the traceback -- https://github.com/joshuadavidthomas/example-django/commit/6e8436eb26e15964e7ef1f6366404adfc06d4dff

I'm getting

Failed to lock ArtifactURL

when generating a lock file, but I can't tell if it is a pex bug or some quicky PyPi problem. It happens with a fresh

~/.cache

and with pip version 22.3 & 23.0.1

# pex3 --version
2.1.129
# pex3 lock create -r requirements.txt  -o foo.lock --interpreter-constraint "CPython==3.10.*" --resolver-version pip-2020-resolver --indent 4 --pip-version 22.3 --target-system linux --target-system mac  --style universal
Failed to lock ArtifactURL(raw_url='<https://files.pythonhosted.org/packages/69/2b/f0ee898c3270d965300ec30b0bf06e062c4cc92f35d17ae6046f429c5067/SQLAlchemy-1.4.25.tar.gz>', normalized_url='<https://files.pythonhosted.org/packages/69/2b/f0ee898c3270d965300ec30b0bf06e062c4cc92f35d17ae6046f429c5067/SQLAlchemy-1.4.25.tar.gz>', scheme='https', path='/packages/69/2b/f0ee898c3270d965300ec30b0bf06e062c4cc92f35d17ae6046f429c5067/SQLAlchemy-1.4.25.tar.gz'). Could not obtain its content for analysis.
# cat requirements.txt 
acryl-datahub[datahub-rest,feast,postgres,snowflake]==0.10.0.2
click>=8.0.0,<9
# TODO: Clarify targets per extra?
feast[aws,snowflake]==0.29.0
snowflake-connector-python[pandas]==2.7.7
typeguard<3
click>=8.0.0,<9
pandas>=1.3.4,<2
pytest>=7.2.0,<8
python-dateutil==2.8.2
pyyaml>=5.1
sortedcontainers==2.4.0
pytz==2021.3
sqlalchemy==1.3.24

Hey all! We've been happily adopting pants for our monorepo setup. One issue we are running into is with a 3rdparty python dependency for which we need to set specific pip options (i.e.

imagecodecs==2023.1.23 --global-option="build_ext" --global-option="--lite"

). In the docs it's stated that these pip options are ignored (for now). Is there a way to work around this? I'd be happy to write up a github issue if that helps

Here's a rough Draft of a new blog post about the latest changes to completely decouple Pants from system Python(s) in 2.16. Comments welcome! https://docs.google.com/document/d/1OfytXhLwp8-YvCYPx6yHm5wje2nDRwv4PFmgv4qeuvo/edit?usp=sharing

How do I make a

python_distribution

dependent on every

python_source

target in a subdirectory?

dependencies=[":pyproject", "mypackage::"],

is throwing an error.

Is there's any way to get more information when pants is running

54.61s Building archipelago.pex with 13 requirements: SQLAlchemy~=1.4.27...

? One of my biggest issues has been around how unpredictable pants performance is. Commands that pants use to run relatively quickly can balloon into what seems like arbitrarily long runtimes after what feels like slight requirement changes. From a developer's point of view running a python command can't have a 5+ minute pants prelude (and its still going)

Hi all! I'm aware that I can do

PANTS_SHA=8553e8cbc5a1d9da3f84dcfc5e7bf3139847fb5f pants --version

to run a non-release version of Pants. Is there a way to set this in

pants.toml

? There's a couple of bugs that I'm working around that have been fixed and cherry-picked to

2.15.x

.

Hi there, I'm just getting started with splitting up a single-project monorepo into multiple projects, which each subproject defining its requirements using a

pyproject.toml

file & the

poetry_requirements

target. Right now I'm just trying to get tests to run (via

pants test src/python/subproject1::

) for one of the subprojects, but I'm getting the following error on an import from a third-party library (

dropbox

, but I think this is likely to happen with a number of third-party dependencies):

ModuleNotFoundError: No module named 'pkg_resources'

I see this resolved issue in the pantsbuild github which seems very similar, but specific to gunicorn. I tried adding

setuptools

to the pyproject.toml file for the subproject and re-ran

generate_lockfiles

before trying the tests again but didn't notice any change. I also tried adding

python_sources(
    dependencies=["3rdparty/python#setuptools"]
)

to the BUILD file for the

subproject1

folder (not sure this even makes sense), and also didn't see any changes. Would anyone be able to point me in the right direction?

When I try to run code with

--debug-adapter

I get

pex.environment.ResolveError: A distribution for scipy could not be resolved for /usr/bin/python3.7.
Found 1 distribution for scipy that do not apply:
1.) The wheel tags for scipy 1.9.3 are cp310-cp310-manylinux_2_17_x86_64, cp310-cp310-manylinux2014_x86_64 which do not match the supported tags of /usr/bin/python3.7:
cp37-cp37m-manylinux_2_28_x86_64
... 484 more ...

My code uses python3.10 and I have that set in my pants.toml as

[python]
interpreter_constraints = ["CPython==3.10.*"]

What do I need to do to get the tool to also use a more recent version of python?

Hi all, I’m trying to create a

pex_binary

target with only the dependencies (including first-party dependencies) of a particular target (actually, a

python_sources

target generator). Is there a way I can tell pants to do this with the

dependencies

field of the

pex_binary

target? The use case is that I have some workflow code which is uploaded to my Docker image by a separate tool (Flyte), so I only want my pex that I’m unpacking into a venv to include 3rd party and 1st party dependency code. Thanks!

How does one work around an error like

native_engine.IntrinsicError: Error hashing/capturing URL fetch response: Downloaded file was larger than expected digest

?

I had a slight misunderstanding in the pants getting started. As an innexperienced curl user I thought that command would execute the install script for pants for me. But I just needed to read the documentation "closelier". I didn't pipe the curl command into a file. I read it as "This curl command will install pants into your ~/bin dir", not as what it actually says. Which is this script will install pants. So I had to pipe it first, chmod, then run it. Would it be fair to update the curl command and add

> install_pants.sh

at the end of it, then on a couple more lines add

chmod +x install_pants.sh

so it's clearer that you're just "downloading" the pants install script? https://www.pantsbuild.org/docs/installation My initial reaction was: "Darn, probably something wrong with my weird RHEL setup" 🙃

New

pants

user. I've added some new backends after installing and initializing pants. What command do I run to get pants to recognize the new backends?

Hello - I'm trying to create an AWS Lambda zip file. Im running my command to create the zip on an EC2 ubuntu instance since my local machine is a Mac. However, I'm getting this error when I run

~/bin/pants package project/lambda_example.py

. Any ideas?

21:23:38.17 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal
  in Create Python AWS Lambda - project:lambda

ProcessExecutionFailure: Process 'Building lambdex.pex from lambdex_default.lock' failed with exit code 1.
stdout:

stderr:
There were 2 errors downloading required artifacts:
1. lambdex 0.1.9 from <https://files.pythonhosted.org/packages/20/89/05c73da172c510299869e904543825222ffa990b719ffaba4bce2d1be0c4/lambdex-0.1.9-py2.py3-none-any.whl>
    ModuleNotFoundError: No module named 'distutils.util'
2. pex 2.1.111 from <https://files.pythonhosted.org/packages/5d/08/89438cc626ec77a6f7dc3ab17cad581d14882a2f51a37d24c8a4c127e7bc/pex-2.1.111-py2.py3-none-any.whl>
    ModuleNotFoundError: No module named 'distutils.util'

Is there a pants library SDK that I can call instead of having to use the cli? I'd like to create new projects and invoke

pants package

from my backend service, which is written in Golang

Do I need to specify a module_mapping (or something else) to get the following import to work?

from langchain.llms import BaseLLM, OpenAI