pantsbuild #general

Channels

welcome

github-notifications

announce

abundant-leather-17386

08/11/2022, 9:09 AM

Hello! I was wondering if any SQL support is already being considered? We use sqlfluff for linting and formatting our .sql files, and it would simplify things if that could be done using pants. If it is not yet on the radar, I wouldn’t mind doing the work myself if someone can tell me how to get started 🙂

🙏 2

bland-father-19717

08/11/2022, 10:57 AM

Is it possible to create base docker image A with skip_push=True and use it as

FROM A

in another docker_image?

narrow-judge-15725

08/11/2022, 12:24 PM

hi everyone. i have a question about the helm backend and dependency inference. 🧵

steep-controller-23208

08/11/2022, 1:38 PM

Hi, I wonder why pants always takes a time while building third-party packages with a message

Building ${n} requirements for ${pexfile} from the lockfiles/python-default.lock

, even though the lock does not change. Is this required process for building pex or is there any way to speeding up this process? Thank you in advance.

bitter-ability-32190

08/11/2022, 4:51 PM

@wide-midnight-78598’s VS Code plugin got my gears spinning VERY hard. I completely neglected all obligations until I could get this PoC for a Pants

BUILD

file LSP. Still hand-waivey, but 🎉

🤯 3

🤩 3

⚙️ 1

bitter-ability-32190

08/11/2022, 5:45 PM

Any reason

coverage-py

doesn't support

lcov

output?

clever-ghost-87030

08/11/2022, 10:02 PM

I've been looking at

./pants generate-lockfiles

in v 2.12 and noticed that the lockfiles generated seem to have this "platform_tag" field in "locked_resolves". From what I can tell, this seems to be dependent on the machine that actually generated the lockfiles. This seems bad from a consistency POV. Is my understanding of this field correct? If so what is it's purpose?

brainy-solstice-27042

08/12/2022, 2:35 AM

What would be the proper way to get pants 1.30 running engine v2 to build another pex as part of the dependency list? The use case is when I build pex A, post build it also requites that pex B was built. We use dot have

:path/to/target:binary

as the definition for the dependency but that doesn't appear to force pex B to be build before pex A is build

limited-insurance-37393

08/12/2022, 1:58 PM

Is it possible to set up a pair of Python build targets that are in the same directory level and one of them depends on the other? Specifically, I have a Python project where I want to package all of the

lib

module as a distribution for publishing to PyPI, and I also have code in a sibling module that depends on the contents of

lib

which I want to package separately. The layout looks like:

Copy code

src/
  concourse/
    BUILD
    lib/
    pipelines/

narrow-judge-15725

08/12/2022, 2:41 PM

sorry, me again. i have a general conceptual question about hermeticity and pants. i just started so bear with me if this is stupid but i’ve seen that pants relies, for example, in having the python interpreter in the machine. this means that technically we could still have the “works on my machine”. i played with bazel and they have the concept of toolchains so one doesn’t need to install anything and it makes sure that the environment (including the tools) is the same always. did i miss anything or is there an explaination for this? thanks!

fresh-cat-90827

08/12/2022, 3:03 PM

I think I’ve seen it somewhere where that it’s possible to treat Pants warnings as errors, but can’t find the global option anywhere in the doc? 😕

👀 1

rich-london-74860

08/12/2022, 3:11 PM

Hello 👋 I’m having an issue where if I run

./pants roots

and

./pants list ::

(or any goal using

::

), then … 1. both commands work locally 2. both work when running in a docker container for the image used in CI/CD 3. in CI/CD,

roots

works as expected, and has the same output as 1 and 2 but

list

gets the error:

Copy code

15:09:10.80 [ERROR] 1 Exception encountered:
  ResolveError: No targets found for the address glob `::`
Do targets exist in those directories? Maybe run `./pants tailor` to generate BUILD files? See <https://www.pantsbuild.org/v2.12/docs/targets> about targets and BUILD files.

Why might this be different in this case?

proud-dentist-22844

08/12/2022, 4:30 PM

Someone in the StackStorm community wants to do development on a raspberry pi. I'm working towards integrating pants, so I'll need a linux arm64 wheel - are there any plans to build that?

bumpy-spoon-55438

08/13/2022, 9:26 PM

Hi, I am writing a custom plugin which a command is invoked in a Process and multiple documents are generated. the problem is I don't know what files and directories will be created beforehand so how to construct the output files for the Process?

nice-florist-55958

08/14/2022, 4:58 PM

With cacheing, is it roughly accurate to say • pantds: invalidates at the file metadata level (e.g., from a

touch -a source.py

), and • pants: invalidates at the file content (SHA hash) level (e.g., from an

echo -e "\n" >>source.py

)? Besides scanning the logs with

-ldebug

option, is there a way to see extra output regarding cache hits from both levels?

wide-midnight-78598

08/14/2022, 5:01 PM

I think I'm late to the party on this one, since somehow I never saw these results until today. https://www.jetbrains.com/lp/devecosystem-2021/cpp/ Comments in 🧵 about how these results relate to my thoughts on the C/C++ backend that I'm puttering along with

acceptable-football-32760

08/15/2022, 12:50 PM

[UPD: ver ==

2.12

] [UPD2: I tried to parametrize the

python_resolve

but then I get another error in 🧵 ] Hello dear community, I'm in the process of creating two resolves (a long-known problem of having gpu+cpu torch dependency) and decided to go via macros for

python_sources

python_requirements

and the likes -- macros which parametrize them like

Copy code

def python_sources_custom(**kwargs):
    kwargs.pop("resolve", None)

    python_sources(resolve=parametrize("gpu", "cpu"), **kwargs)

All uneventful until I hit the grpc

proto

generation and it seems that there's something off with them: • on the one hand,

protobuf_sources

target generator does not have anything like

resolve

field (github) -- which makes sense provided that proto file is not bound to any specific language and resolve mechanism is python-specific -- so I can't actually parametrize them • on the other hand, I hit an error for dependency resolver

Copy code

UnownedDependencyError: Pants cannot infer owners for the following imports in the target some/path/to/file.py@resolve=gpu:

  * some.import.of.this.file_pb2_grpc (line: 1)

These imports are not in the resolve used by the target (`gpu`), but they were present in other resolves:

  * some.import.of.this.file_pb2_grpc: 'cpu' from some/import/of/this/file.proto

• a.k.a. "despite

.proto

file itself being not subject to resolve, pants naturally wants theresulting

pb2_grpc.py

file to be resolved" It seems to be a dead-lock 😅 or am I missing something?

rich-london-74860

08/15/2022, 3:58 PM

Hello 👋, new to pants in the last week so this may be a stupid question. This is a specific question about deploying python packages, but also a general question about how things should be done. I need to deploy a python package to a private repository and the credentials for that private repository are stored in environment variables (For the sake of this question, please assume how these credentials are stored may not change). In general, it seems to me that with the exception of alterations to

pants.toml

, pants is not really designed to utilize environment variables, everything should be in a file, whether it’s check-in into the repo or a configuration file on the machine. I was able to workaround this in 2 ways. 1. One way I have this working is to use

./pants package

to produce the

*.whl

files and then use a

twine upload

(which uses environment variables to set the pypi target -

TWINE_REPOSITORY_URL

TWINE_USERNAME

TWINE_PASSWORD

) to publish those files (For the sake of this question, please assume that only python packages need to be published, meaning this would fulfill all of our needs). However, this does not use

./pants publish

and does not use the

repositories

property of the

python_distribution

target, which seems wrong to me. 2. Another way I had this working was to write a script that writes

~/.pypirc

based on credentials in environment variables. That script is built into a

pex_binary

target and run before

./pants publish

. When

./pants publish

runs, it picks up

~/.pypirc

. However, it seems like

pants

is designed to have all files either checked-in, a persisted fixture of the environment (the CI/CD worker instance has a

~/.pypirc

file saved on disk), or the product of a target specified in a

BUILD

file. In this case

~/.pypirc

is not any of these, so this feels very hacky to me.

rhythmic-airport-91451

08/15/2022, 4:21 PM

Hey All 👋 I'm adopting a project that utilized Pants for Python and am trying to learn while doing. Just had a couple of questions to help troubleshoot. 1. I'm currently on an M1 mac and the project was developed on an intel based mac and we were using python 3.8.13 to do our development and configure Pants against (using Pants v2.10). I'm having some issues trying to use 3.8.13 and 2.10 respectively where I'll receive the ERROR: No matching distribution found for pantsbuild.pants==2.10.0 while trying to run any Pants command, though this does work when I swap to Python 3.9.x. Just was wondering if this is an issue around arm not being supported for anything below 3.9. 2. If I understood correctly, Pants only focuses on the directory and subdirectories that it belongs in and doesn't recognize anything outside of it. I'm trying to utilize a common directory for doing configuration setups and am having a hard time trying to get Pants to notice that library in the BUILD file. Just looking for any suggestions on how to recognize another directory or file in the file structure outside of directory the BUILD file is in.

rhythmic-glass-66959

08/15/2022, 6:39 PM

Hi, I'm trying to copy python sources into a docker image. I added the

python_sources

target to the

docker_image

dependencies, but when building the docker image, I got an error message that the

file does not exist

boundless-zebra-79556

08/15/2022, 9:01 PM

Hi all! I've filed issue https://github.com/pantsbuild/pants/issues/16494 wrt adding

--pull=False

to docker CLI when doing

docker_image

. This will come handy when build process is multi-stage and each target can assume one stage of the building process. Seems like a straightforward change and if we are ok with the idea, I would like to take a stab at adding

pull

boolean field to

docker_image

rule. What do you all think?

✅ 1

busy-vase-39202

08/16/2022, 12:20 AM

Thank you to everyone who participated in the Pants Team Meetup today. Great conversations. Much appreciation to @hundreds-father-404 @bitter-ability-32190 @happy-kitchen-89482 @witty-crayon-22786 @flat-zoo-31952 especially. Followup items are: • Josh Cannon will open thread(s) for specific bug smash questions that need decisions/actions • Josh Reed will open a design doc and GH issue re deprecation policy • Josh Reed will open thread re bikeshedding the “experimental” label • Carina will solicit team members for best email address • Stu will open thread re whether and how to hold meetup retros

👖 6

nutritious-minister-3808

08/16/2022, 1:01 PM

I ran across an uncommon use case after successfully migrating to pantsbuild. This is probably more of a pex question than a pantsbuild question but I figured I'd start here as I can't seem to find a similar platform for pex related questions. Is it possible to use a pex file to execute a script in
-i
interactive mode? There is a script in a docker container that was previously executed using

/path/to/venv/bin/python -i script.py arg1 arg2

. Now with pantsbuild, the

/path/to/venv

no longer needs to exist because everything else in the container is packaged using pex. I have considered leaving the current venv there as well as possibly using a modification of this recipe to make a new venv but I was wondering if there was some possible trickery with pex that I was overlooking?

ancient-breakfast-45303

08/16/2022, 1:44 PM

I saw that pex is going to get a newer pip, but I recently learned that pip changed the behaviour of find-links/extra-index (https://github.com/pypa/pip/issues/9959) is it going to be possible to work around that with pex or is that something I will have to just work around? I have a test case to share if it's something that's considered a bug, I just want a wheel that is available both on pypi and find-links, with find-links having a larger build number to be preferred

✅ 1

ripe-cpu-85141

08/16/2022, 4:04 PM

Hello, I'm fixing dependencies in my project as right now, I'm pulling everything for everything and Pants seems to have some issues inferring some dependencies with pytest and pytest-mock. I'm having lines like

Copy code

@pytest.fixture(autouse=True)
def dont_do_this(mocker):
    mocker.patch("my.module.class.method")

and Pants doesn't seem to add

my/module.py

in the dependencies even with

Copy code

[python-infer]
string_imports = true

Am I missing something or should I manually add those deps in my BUILD file?

bitter-ability-32190

08/16/2022, 4:30 PM

🚲 🏠 On some an aspect of eliding

update-build-files

into `fmt`/`fix`. • Let's say you're formatting your build files with

black

. Running

./pants fmt --only=black ::

with `black`en your Python code AND your

BUILD

files. How does that feel, as a user? The alternative is to make some "special" key (like

build-files-black

or similar), which feels worse to me. • Do we need/want options for "only BUILD files"/"everything but build files"? Those are possible, but tricky to find a good name for IMO.

freezing-lamp-12123

08/16/2022, 5:25 PM

okay, so i have a project with a mix of python, go, jsonnet and a whole giant heap of bash scripts that are written with the gnu variants of coreutils + friends in mind now — i’m almost settled on for mac laptops to provide a complete user installed shell + coreutils setup through an isolated homebrew instance (talk me out of it please 😂) any pointers on how to provide such a dependency conditionally based on pants is running on laptop va ci? or is there some other tree i should be barking up at? 😁 thanks in advance 🙏

refined-addition-53644

08/16/2022, 7:57 PM

Hi, It’s not very clear to me how the

complete_platforms

option should look like in

pex_binary

The docs for pex says

The complete platform can be either a path to a file containing JSON data or else a JSON object literal.

On the other hand, docs for pex_binary says

Complete platforms should be addresses of
file
targets that point to files that contain complete platform JSON as described by Pex

fresh-cat-90827

08/16/2022, 9:37 PM

hey folks — do we plan to build and publish to PyPI Pants for Python version 3.10?

plain-truck-80265

08/17/2022, 5:15 AM

Hey all 👋, I am using

pants publish

to push Docker images into ECR. It works great, but the problem of ECR is that you must have repo created before you push the iamge (https://github.com/aws/containers-roadmap/issues/853) , otherwise you see couple of retryings. Is there any elagant way how to run some command before

pants

do a docker push? Thank you