I'm running into trouble getting certain libraries (

scs

in particular) to install on a mac. It looks like the issue is with pants picking up a universal binary and attempting to build an arm64 version of the library that we don't need and failing. I've been able to install in docker, but still struggling on mac native. One of my co-workers was able to get it to work by forcing an

interpreter_constraints

to a version that doesn't have a universal binary installed. This feels like a fairly unmaintainable hack, so I'm hoping there is a better way

Different question: I believe I am struggling with pants crashing when trying to find standard packages ie

attrs

in our internal

artifactory

. My best guess is that our artifactory is returning an empty list or an error that is then causing pants to fail. Is there a better way I should be handling this? I'm currently adding things with

[python-repos].indexes.add

re: Remote Caching (and eventually execution). We're just using a self hosted TeamCity at the moment and Pants + Remote Caching could really speed our build up. If we where to pick one of the suggested compatible servers: Build[Barn|Farm|Grid] what are folks opinions on which we should pick? Minimum maintenance (but self hosted) and speed are our main concerns and right now remote caching is our goal but we don't want to rule out remote execution.

Is there a built-in way for Pants to give a list of files that it doesn't currently cover? I'm thinking of ways to prevent, say, adding a new shell script that and forgetting to add / modify a BUILD file to make sure that it gets linted, formatted, etc.

I'll try to explain what I expect to do with Pants, as I am having some issues in getting started (but maybe my approach is the issue). We are moving a bunch of python (and cython) project to a monorepo structure, I read some articles about different approaches and the way we want go for involves having multiple "project roots" in subfolders. Each project root has its own pyproject.toml, poetry.lock and set of dependencies. Does pants support this setup? I couldn't find any examples or documentation to answer this question, maybe I'm just blind though 😄. I tried this very quickly, so maybe I'm missing some obvious stuff,

./pants dependencies projects/example

doesn't return any dependency, I tried both

poetry_requirements

and

python_requirements

. If pants can indeed support this setup, do you have any examples of open source monorepo?

i’m using monterey on an M1 pro and i get this error when upgrading pants to 2.7.1 (2.7.0 has been working normally). has anyone else experienced this?

martim@Argon repo % poetry update
Updating dependencies
Resolving dependencies... (22.5s)

Writing lock file

Package operations: 0 installs, 8 updates, 1 removal

  • Removing requests-unixsocket (0.2.0)
  • Updating jupyter-server (1.11.1 -> 1.11.2)
  • Updating botocore (1.22.8 -> 1.22.9)
  • Updating yarl (1.7.0 -> 1.7.2)
  • Updating boto3 (1.19.8 -> 1.19.9)
  • Updating isort (5.9.3 -> 5.10.0)
  • Updating regex (2021.10.23 -> 2021.11.2)
  • Updating pantsbuild.pants (2.7.0 -> 2.7.1): Failed

  RuntimeError

  Unable to find installation candidates for pantsbuild.pants (2.7.1)

  at ~/.poetry/lib/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link

hey! i'm quite new to pants and am using it for a couple of weeks now in my project. i'm using python and primarily lambda targets to build my apps.. however, it turned out that pex based lambdas seem to take quite long to bootstrap when having large deps like tensorflow, so i'm looking for alternative ways to package my lambda functions. is there any convenient way in pants to create some kind of "fat wheels" consisting of all required dependencies (resolved automatically), without having to rely on pex? thanks in advance!

Hello again, I've been busy with other projects but still wanted to follow up on Enigma's proof-of-concept with Pants so far. Personally, I am sold and already used Pants for a tools repository. It is rather nice to tell people: Just clone and run

./pants run source/tool

(well, and get that GitLab token). As y'all suggested, I got documentation building with Sphinx working by creating a PEX target that includes Sphinx, its plugins, and the repository's source code as dependencies. The one snafu and remaining issue is that Sphinx won't find the source code. I had to explicitly:

sys.path.insert(0, os.path.abspath('../libraries'))
sys.path.insert(0, os.path.abspath('../tasks'))

in Sphinx'

conf.py

. I'm not sure why. But it won't work without.

The second issue is conceptual: We currently have one repository per library and build up pipeline stages by declaring the internal and external dependencies in

requirements.txt

. For some projects or some branches, we also pin internal dependencies. With a Pants-based monorepo, it makes sense to develop and test against the union of internal libraries, i.e., the source code as is. I can see how to cut different libraries from the monorepo as well. But I don't see a good solution to distributing libraries with pinned internal dependencies. Essentially, I'm asking how can I layer a packaged and possibly pinned view on top of the all sources in a big soup view of the monorepo with Pants. Any help in resolving this last remaining issue would be fantastic.

is there a way to keep the

sources

field default, but w/ some extras?

Hello 🙂 Is there a way to package my code plus all dependencies in a folder or archive? I was looking at the archive target, but as I understood it, its not a package target itself and wont add dependencies. Sadly I cannot use PEX.

Is there an easy way to clean up (or even just find) all of the files left behind by running with

--no-process-execution-local-cleanup

. I ended up filling up quite a bit of my disk yesterday running repeatedly to debug

Got a strange issue. We build for both ARM and intel. Today I submitted a PR that didn't change anything about our Pants version (2.6.0) nor did it change any of the dependencies in our

constraints.txt

. In fact, it contained only a few shell scripts. But the CI failed with:

WARNING: Discarding <https://files.pythonhosted.org/packages/01/1b/d3ddcabd5817be02df0e6ee20d64f77ff6d0d97f83b77f65e98c8a651981/numpy-1.18.5.zip#sha256=34e96e9dae65c4839bd80012023aadd6ee2ccb73ce7fdf3074c62f301e63120b> (from <https://pypi.org/simple/numpy/>) (requires-python:>=3.5). Command errored out with exit status 1: /builds/companionlabs/companionlabs-central/python/.pants/named_caches/pex_root/venvs/short/d492f796/bin/python3.7 /builds/companionlabs/companionlabs-central/python/.pants/named_caches/pex_root/venvs/short/d492f796/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/process-executionM0T3Ob/.tmp/tmp6sx3gu76 Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement numpy
ERROR: No matching distribution found for numpy

This is the same version of numpy we've been using successfully on ARM for months. And we're using Python3.7 but it seems to say it's discarding

numpy

as it requires

python:>=3.5

which would seem to be fine. Any ideas?

Is there a way to get shell autocomplete working with pants?

If I've got a thing with a deep nesting of packages, but everything beyond the 2nd level is just one 'unit', is there a best way of dealing with this? Is it just set the

sources

of the

python_sources

to

***/**.py, !**/test_*.py

?

Any one have any hints on what be needed to get pants to work with

pybedtools

?

Hello, I am looking at pants to use on a mono repo. The repo contains a few projects (~7 at the moment. Looking at how pants work, I am wondering if it is possible to have projects depending on older versions of other projects in the repo. The reason for this use case is that some projects have dependencies that are not compatible with some new projects that are coming up. The common libraries has to upgrade with these new libraries. I am not experienced with mono repo yet. I would be happy to have feedback from more experience users.

Hello 👋 I'm wondering if there is any way to configure a particular

python_tests

target to to run in series? Using pants 2.7

Hi guys, I hope you can help me with the issue we’ve faced :) We use pants 2.7.0 for our mono repo with about 20+ projects and since we switched from managed Github Actions runners to self-hosted we notice that pants is consuming a lot of memory to run python_tests. When the limit was 4Gib the pods were often killed. Since then we’ve increased the limit to 8Gib and now it looks pretty stable, but sometimes the consumption goes as high as 85%=~7Gib and as the codebase is going to grow together with the tests, I have some concerns. I believe we face a combination of https://github.com/pantsbuild/pants/issues/12662 and https://github.com/kubernetes/kubernetes/issues/43916 here, and there are a few workarounds around this issue, but maybe I’m missing/misconfigured something on the pants side as well. I would appreciate any help :)

Is it possible to use $HOME or ~ in extra_env_vars?

Hey, moving some of my team's code( which was in its own pants monorepo) into a new pants codebase and trying to make sure IntelliJ plugin is able to import the project. Using

IntelliJ 2020.2.4

,

pants plugin 1.16

, with pants

1.30.5rc1

. It sometimes will work and some other it will run way too long ( before I cancel it). When it does run long and I cancel it I can see the following output

pants --spec-file=/private/var/folders/jl/2530nrx54zl8q3bmz7bcg3yc0000gn/T/pants_target_specs1366.in --no-quiet --export-available-target-types export --output-file=/private/var/folders/jl/2530nrx54zl8q3bmz7bcg3yc0000gn/T/pants_depmap_run2768.out --formatted
Exit code: 120

16:24:41 00:00 [main]
               (To run a reporting server: ./pants server)
16:24:47 00:06   [setup]
16:24:47 00:06     [parse]
               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> unpack-wheels -> deferred-sources -> native-compile -> link -> gen -> jvm-platform-validate -> pyprep -> export
16:24:48 00:07   [bootstrap]
16:24:48 00:07     [substitute-aliased-targets]
16:24:48 00:07     [jar-dependency-management]
16:24:48 00:07     [bootstrap-jvm-tools]
16:24:48 00:07     [provide-tools-jar]
16:24:48 00:07   [imports]
16:24:48 00:07     [ivy-imports]
16:24:48 00:07   [unpack-jars]
16:24:48 00:07     [unpack-jars]
16:24:48 00:07   [unpack-wheels]
16:24:48 00:07     [unpack-wheels]
16:24:48 00:07   [deferred-sources]
16:24:48 00:07     [deferred-sources]
16:24:48 00:07   [native-compile]
16:24:48 00:07     [conan-prep]
16:24:48 00:07     [conan-fetch]
16:24:48 00:07     [c-for-ctypes]
16:24:48 00:07     [cpp-for-ctypes]
16:24:49 00:08   [link]
16:24:49 00:08     [shared-libraries]
16:24:49 00:08   [gen]
16:24:49 00:08     [protoc]
16:24:49 00:08     [thrift-java]
16:24:49 00:08     [thrift-py]
16:24:49 00:08     [py-thrift-namespace-clash-check]
16:24:49 00:08     [grpcio-prep]
16:24:49 00:08     [grpcio-run]
16:24:49 00:08     [scalapb-gen]
16:24:49 00:08     [jooq-gen]
16:24:49 00:08   [jvm-platform-validate]
16:24:49 00:08     [jvm-platform-validate]
16:24:49 00:08   [pyprep]
16:24:49 00:08     [interpreter]
16:24:49 00:08     [build-local-dists]
16:24:49 00:08     [requirements]
16:24:49 00:08     [sources]
16:24:49 00:08   [export]
16:24:49 00:08     [export]
               Waiting for background workers to finish.
16:25:00 00:19   [complete]
               SUCCESS
20:24:44 [INFO] initializing pantsd...
20:24:47 [INFO] pantsd initialized.
20:24:47 [WARN] Unmatched glob from products/datalogue/utilities/test/resources:resources's `sources` field: "products/datalogue/utilities/test/resources/*/**"

Seems like its getting stuck after initializing pantsd but have no clue how to debug this or find the root cause. Given this running all within IntelliJ , I am not sure how to add any debug logs to the pants process. Any tips on how I can go about finding the issue ? Can also share the intellij logs if that can be helpful

Hello! I've got a few, hopefully simple, questions that I was hoping someone could point me in the right direction for. Question 1: I have an existing scrapy project I'm trying to put into a pex. Before pex, I would run

scrapy runspider myproj/spider/myspider.py

but now I've got:

python_library()

pex_binary(
     name="bin",
     entry_point="scrapy.cmdline:execute"
)

which will execute scrapy but I can't figure how to pass the

runspider myproj/spider/myspider.py

Question 2: Is there a python plugin (or command I missed) that will generate a virtualenv for local development? In the example above I can use pants to generate a pex, but can I use pants to build a virtualenv so I can hack on the project as well? I know I can take a pex and turn that into a virtualenv but was wondering if this is built in

I think we should maybe make it easier to onboard using Pants only for formatters/linters w/ a language. Right now, you have to set this:

[GLOBAL]
backend_packages = [
  "pants.backend.shell",
  "pants.backend.shell.lint.shellcheck",
]

Should maybe just require

pants.backend.shell.lint.shellcheck

? Thoughts welcomed! 🧵

how are you meant to turn something that runs from another script (e.g.

gunicorn

) into a

.pex

? setting

script="gunicorn"

works... but doesn't allow specifying the

myproject.wsgi

argument.

Is there a way how Pants can be used to check for circular dependencies between packages? For instance, during the last release of our repo (containing multiple interdependent Python packages) I accidentally introduced mutual dependency between two packages (not between modules) and I would like to add some kind of check to prevent this to the CI. Any idea? :-)

is the only difference between the 2.7.0

python_library

and the 2.8.0

python_sources

just the name? it /feels/ like everything works differently

hi 👋 is there a way in Pants to run code before adding a dependency to the PEX file? similar to setup.py when running pip install

How should one go about handling building a Python C Extension Module? Is there a mechanism in-ecosystem to do this, or is it more of a "do-it-outside-of-pants" thing?

hi yall, I'm wondering if there is a way to build distributions with overlapping file ownerships? We have to use wheels due to limitations in databricks. I want to build "fat wheels" for each job. But pants is not letting me to lnclude the same file twice in two different wheels.

Hi team, for snapshot jvm dependencies can the url point to a remote file location ? The documentation only mentions local files but wondering if its possible to point the BUILD to look for a jar file in S3