Does anyone have a VS Code stub or something for

BUILD

files? I like to have them set as the Python type to get autoformatting, but it's a sea of red because

python_sources

and such aren't defined

We have a docker-compose that uses approach #1 in https://blog.pantsbuild.org/pants-pex-and-docker/ of "What you're probably doing today"; i.e. all the Python code for an app is copied to the container, pip install is run there, and then a shell script is run to fire up Flask. I'd like to use Pants's support for building Docker images such that our docker-compose file would just refer to those images (I think). But we also depend on using PyCharm to debug code when it's running in Docker. Does that seem possible still if docker-compose is just running images built by Pants? I am thinking "no" because I don't know how PyCharm's remote interpreter would be able to map local source code to the corresponding code in a running Docker container.

I would like to be able to use wildcards when declaring dependencies in a BUILD file. For context we have a bunch of django apps in one repo which are part of one django project. When running migrations I would like to be able to depend on

src/django_apps/::/migrations

, but it seems that I can't do that in a BUILD file. Is there a good way around this? Or will we need to manually manage that list?

Cache boost idea for formatters when linting 🧵 It should be possible to have both

fmt

and

lint

use the same process (the args used for formatting) for both

fmt

and

lint

.

fmt

then applies the changes to the workspace while

lint

diffs (optionally outputting the diff if requested). Then

./pants fmt lint ::

is a no-op in

lint

for formatters 🎉 The risk is meaningful

stdout/err

being removed from the tool, but the gain is perf through caching, consistency, and plugin simplification.

Should this work? `pants.toml`:

...
[docker]
build_args.add = [
    "BASE_PYTHON_IMAGE=foo",
]

`pants.ci.toml`:

...
[docker]
# Expose aws auth to docker.
build_args.add = [
    "AWS_DEFAULT_REGION",
    "AWS_ACCESS_KEY_ID",
    "AWS_SECRET_ACCESS_KEY",
    "AWS_SESSION_TOKEN",
]

`Dockerfile`:

ARG BASE_PYTHON_IMAGE
FROM ${BASE_PYTHON_IMAGE}
...

Running in CI:

$ export PANTS_CONFIG_FILES=pants.ci.toml
$ ./pants package docker/image:target
...
  failed to solve with frontend dockerfile.v0: failed to create LLB definition: base name (${BASE_PYTHON_IMAGE}) should not be blank
...

Any ideas on building a JVM JAR file with only the compiled classes code but without the 3rd party dependencies? The situation I’m trying to deal here is some dependencies on Akka, since each Akka JAR file has a

reference.conf

file at the top-level, when trying to pack the code that needs Akka using

deploy_jar

we get file colissions (as it finds the same file name more than once). So this either requires having a target that allows to package a thin jar, or use something similar to what SBT does (in

sbt-assembly

) by having Merge strategies, not sure if anyone here has thought on other ways of solving this.

hi! not sure if anyone else has had this problem before: when I run a pants build with a

fastparquet

dependency, it errors out on

numpy

being installed already:

FileExistsError: [Errno 17] File exists: '/private/var/folders/5c/tp4p_z3d2bnc6fb68lth0d4w0000gp/T/process-executionyhk5sS/.tmp/pip-req-build-vkux75ls/.eggs/numpy-1.14.5-py3.7-macosx-10.16-x86_64.egg'

this is python

3.7.10

, pants

2.9.0

.

fastparquet==0.7.1

meanwhile I can

pip install fastparquet==0.7.1

just fine, but it's using a cached wheel. it seems to be a known problem for fastparquet 0.7.1 (github link) but I don't know what exactly the solution is

Hi, I'm playing around with

experimental_shell_command

to generate c++ extensions using cmake and pybind11 for python, the build file I'm using right now:

shell_sources(name="build-c-exts", sources=["./**/*"])

experimental_shell_command(
  name="c-deps",
  command="./build.sh",
  log_output=True,
  tools=["make", "cmake", "cat", "bash", "env", "chmod", "gcc", "g++", "as", "ld"],
  dependencies=[":build-c-exts"],
  outputs=["results/", "logs/my-script.log"],
)

But I get an error that it can't find

Python.h

(I'm using pyenv), error in the thread ⬇️

Is there any way to see what triggered this change

10:59:31.12 [INFO] Remote cache/execution options updated: reinitializing scheduler...

?

Hi! I am working on a python project and have adopted pants to manage the monorepo I am building. In order to deploy the application I am using docker and k8s. As pants has an awesome integration with docker I am trying to build images using pants

docker_image

. The image is building successfully, but when I run a container using the image, it is not able to serve requests to the host (made sure that I am forwarding the correct port). I am using docker on WSL2, and have pasted pants'

BUILD

file and the

dockerfile

I'm using to build the image in the thread. Any ideas on how I can fix this? 🙂

We use "--no-binary" with pip today to adhere to the psycopg2 guidance of not using psycopg2-binary in production but rather building it - https://www.psycopg.org/docs/install.html#psycopg-vs-psycopg-binary . Is there a way to achieve this with Pants?

Let's say I have a program that makes a class that can be used as a context manager. And I want to run that like

with MyClass():
   while True:
      time.sleep(1)

When I run this, I see

MyClass.__enter__

called, but if I Ctrl+C, I never see

MyClass.__exit__

called. This only happens with Pants. If I run my program as

python my_prog.py

, then I see

MyClass.__exit__

successfully called. Is there any way to recover this behavior? It makes sense to me why this is happening, but I'd really like to make sure I can gracefully shut down things.

AFAICT, pex_binaries take env variables from the surrounding env. Is there a way to set extra env vars in a BUILD file?

do we have a way to validate sources inside the

BUILD

files to see if they would raise any warnings when target is being used?

Hi Pants team. I am new to Pants and I am trying to get it working inside my company environment. When running

.pants --version

I encounter the following error.

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1122)'))': /simple/virtualenv/

I think this warning comes from

"${python}" "${pex_path}" -r requirements.txt -c virtualenv -o virtualenv.pex

In my company, we have artifactory mirrors /simple. I am wondering how to config pants to use that?

Is it possible to choose a docker registry from cli? We have separate prod and non-prod registries, and the CI pipeline will build images for one or the other depending on tags/branches/etc… Perhaps separate config.ci.prod.toml and config.ci.non-prod.toml with different default registries would be simplest?

An interesting observation. Having a

pex_binary

declaration only, and if the

entry_point

file is not listed as

sources

, the PEX file is still produced, but isn’t usable since the module is absent.

Just wanted to share this image. The first pipeline is a pants-enabled pipeline, the second is our classic pipeline. We still have a long ways to go (there's still a bunch of ways that following transitive deps will hit every test 😂 ), but this is a great start. And it's a great feeling knowing we have a way out of this hole. Thanks to the Pants team!

Is there a way to use globs in dependency declarations? Like... perhaps via macro? Context in 🧵

Hi! anyone knows where Pants stores the wheels it builds as it resolves dependencies?

I'm trying to configure pants remote execution to be able to build a docker image using buildbarn running on kubernetes. For this configuration I am using a dind sidecar. When runner docker in the runner everything is working as expected although when I submit a pants job its not finding the docker daemon. I think this is because pants isn't respecting the DOCKER_HOST environment variable on the remote executor, can any confirm whether this is the case or not. And if there is a way to set the environment variables used for remote execution when running pants?

Hi! Curious if any MacOS Pants user ever tried and succeeded to pass a DYLD environment variable to a Pants process (e.g. a Python test)? • I’m trying to pass

DYLD_FALLBACK_LIBRARY_PATH

. I set it in

test.extra_env_vars

in

pants.toml

, set the environment variable and then call

./pants test ::

. • But it looks like MacOS System Integrity Protection is preventing me to pass the DYLD environment variable to

./pants

, because it’s running a

/bin/bash

binary, which is protected by the SIP. • I understand this is not a Pants issue, but curious if anybody had found a workaround? • Note: the issue has been documented in several StackOverflow questions, but the workarounds are not super practical (e.g. copy the bash executable outside of

/bin

or

/usr/bin

)

FYI two new tooltips added to https://www.pantsbuild.org/v2.10/docs/python-linters-and-formatters on benefits of Pants with fmt and lint, consistent interface + concurrency. Including some really cool improvements made to concurrency by @witty-crayon-22786 in Pants 2.10. Feedback welcomed!

Also FYI new "good first issue" opened: https://github.com/pantsbuild/pants/issues/14711. "Fix confusing help message for sources field." It uses the Target API, not the Rules API, and is hopefully approachable

What would be the best way to use pants as pre-commit hooks for linting and formatting? Just write a custom bash script to call pants ?

Is there a way to add poetry_requirements to the dependencies of pex_binary? （For more information, see thread.）

Hey all, not sure if this is the right channel, but I’ve been doing C++ for about 12+ years. I’ve played around with Rust every now and again over the past couple years, but never got stuck on it. Does anyone have a good resource for learning Rust from C++?

I wanted to compare the performance of Pants dependency inference to perhaps some naive ways of doing it myself, and I'm getting quite better performance with a single-threaded AST scanning method. I know this method doesn't have anywhere near the number of features as Pants does, but I want to better understand why my silly script (12-15 sec) performs so much better than Pants (30-45 sec) for the same codebase and queries? I assume its something involving cache, IO, or task scheduler overhead, but I'd like to hear y'all's input. More details in 🧵

A new

good first issue

that could speed up Pants a decent bit & should be approachable. https://github.com/pantsbuild/pants/issues/14719

Optimize Pants's data structures by not sorting as much (FrozenOrderedSet and FrozenDict)

when bootstrap pants using

./pants --version

, I see the following error

WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)'))': /simple/virtualenv/
ERROR: Could not find a version that satisfies the requirement virtualenv==20.4.7 (from -r requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for virtualenv==20.4.7 (from -r requirements.txt (line 1))
.pex/venvs/32a5dda4259a11b7add428d3f3e5656078dd2fe0/f28b3dbba3c9dae1b4357adde5b079b8b3ca9fac/pex --disable-pip-version-check --no-python-version-warning --exists-action a --use-deprecated legacy-resolver --isolated -q --cache-dir .pex download --dest /tmp/tmpwm2e_tsu/data.user.miniconda3.bin.python3.7 --requirement requirements.txt --index-url <https://pypi.org/simple> --retries 5 --timeout 15

Is there a way to pass other --index-url?