Where does pants look for code? Does it look in the current git-repository scope? I have a 'master' repository with all the micro-repos checked out into it (controlled by

myrepo

or

mr

). under

/libraries

,

/services

etc. But despite setting

[source]
root_patterns = [
    "/scripts",
    "/libraries",
    "/services",
    "/devops",
]

./pants count-loc '*'

doesn't find anything that isn't a file in the root

The following warning message is displayed when the pex root directory is not writable:

PEXWarning: PEX_ROOT is configured as /Users/<username>/.pex but that path is un-writeable, falling back to a temporary PEX_ROOT of /private/var/folders/cf/z3ktw6dn467gvm24fgx8ft6c0000gp/T/tmp50005su_ which will hurt performance.

I am curious about the reference to the performance being reduced. Is it because artifacts like installed wheels cannot be re-used across pex builds or is it because of something else?

https://pantsbuild.slack.com/archives/C046T6T9U/p1620511488406100?thread_ts=1620315506.324900&cid=C046T6T9U Ping for some help - Thanks!

Hi folks, PyCon US week is finally here, and Pants has a strong presence this year! • On Friday at 9:15 AM PDT I’ll be speaking about some of the cool and unusual ways we leverage Python in the Pants execution engine, in “Creating extensible workflows with off-label use of Python”. • On Friday at 1:00 PM PDT @hundreds-father-404 will be speaking, in Spanish, about our experience interoperating Python and Rust, in “Cuándo usar extensiones nativas en Rust: rendimiento accesible y seguro”. • On both Friday and Saturday from 1100 1200 PDT we’ll be demoing “Fast, Friendly Python Builds With Pants” in the virtual expo hall. Also, our company, Toolchain, was selected as one of a handful of startups to be featured in Startup Row, a showcase of new ventures and technologies of particular interest to the PyCon audience. We’ll be demonstrating OSS Pants, as well as our SaaS remote caching solution. Swing by our virtual booth any time to chat! You can still register for PyCon at https://us.pycon.org/2021/registration/information/. It’s only $100 for an individual registration or $150 for a corporate registration, and proceeds benefit the Python Software Foundation, and therefore all of us. Perhaps your employer will reimburse the cost of a ticket? Please tell your friends and co-workers about our talks and demos, and come find us online at the conference!

Pycon ticket is as low as $50 for students and academics!

Hey - I'm trying to add some type stubs to my build and having issues. I've added a

type_stubs

directory to the root of my repo and added the following to my `pants.toml`:

[source]
root_patterns = ['/', 'type_stubs']

I added a

BUILD

file in `type_stubs`:

$ cat type_stubs/BUILD 
python_requirement_library(
   name = 'numpy_stubs',
   requirements=[
      'numpy-stubs@ git+<https://github.com/numpy/numpy-stubs@c49d2d6875971a669a166ea93ef998911af283a1>'
   ]
)

That didn't cause any errors but mypy didn't pick up the stubs for numpy so I also added this to my

BUILD

target that uses `numpy`:

python_library(
   dependencies=[
      '//type_stubs/:numpy_stubs'
   ]
)

which causes the following error:

Exception message: 1 Exception encountered:

  InvalidSpecPath: Address spec has un-normalized path part 'type_stubs/'

Removing the leading

//

didn't help either. What am I doing wrong? And, is there an easier way to add stubs for things like numpy?

hmmm.. Just re-tried removing the leading

//

and now it works. I swear it didn't before 🧌

Unfortunately if I now get:

16:45:19.88 [WARN] Ignoring `[python_setup].resolve_all_constraints` option because constraints file does not cover all requirements.

which I think means my

constraints.txt

is being ignored (I didn't quite grok the docs). That seems unfortunate. Note that the stubs are

git+http

style so I can't freeze them even if I did put them in the

<http://requirements.in|requirements.in>

. Is there no way to get these 3rd party type stubs but also keep the

constraints.txt

working?

And, even better, is there no way to use stubs like this and not have to make them an explicit dependency? The docs for stubs (https://www.pantsbuild.org/docs/python-typecheck-goal#type-stubs-pyi-files) seem to say I shouldn't need to do that but they're not discovered unless I add them as an explicit dependency.

Our options docs for v1.30 - vPresent (https://www.pantsbuild.org/docs/options#list-values) currently leave out the implicit scalar add form supported by flags and environment variables. The v1 docs explain this form though (https://v1.pantsbuild.org/options.html). Is the v2 doc omission by design for uniformity / decreased complexity of explanation or just an oversight? (where scalar form means

--a-list-option=a

appends a to the the list option just as the vector form

--a-list-option="+['a']"

does)

Hi all! This is a big week for Pants, and to make it even bigger, my company, Toolchain, is throwing open the doors to the beta of our remote caching system. If you're using Pants and want to speed up your desktop and CI builds with shared, remote caching, just fill out this form: https://toolcha.in/beta (or DM me for more details)

That typeform link again: https://toolcha.in/beta

Figure I'll throw this out there as another solution for varying django settings on a per-app basis within a monorepo. We define a

test.py

settings module, which does a wildcard import on a

shared.py

setting file and then does a few overrides. This is working for me and it is a much simpler version of what django does internally:

from django.conf import settings
from my.app.settings import test as test_settings


def pytest_configure():
    """ Configure django settings """
    settings.configure(**{key: getattr(test_settings, key) for key in dir(test_settings) if key.isupper()})

I just added

skip_pylint=True

to a lot of BUILD files, but pylint is still reporting errors on py files in those directories. Any idea what I'm missing? https://github.com/st2sandbox/st2/commit/aa2ba3a2bb8540fbc80f82d644ba2c7ffad2d3a0

Can I use environment vars in

[pylint].args

?

[pylint]
args = [
  "--jobs=${PYLINT_CONCURRENCY:-1}",
]
[subprocess-environment]
env_vars.add = [
  "PYLINT_CONCURRENCY",
]

It looks like I can't use shell syntax, so is there another way?

pex: error: option --jobs: invalid integer value: '${PYLINT_CONCURRENCY:-1}'

🎉 I got pylint running reasonably well under pants for https://github.com/st2sandbox/st2/tree/pants So,

./pants lint ::

runs black, flake8, and pylint on the appropriate files now and it completes in a reasonable amount of time (pylint takes a couple minutes) Next step "Set up tests": https://www.pantsbuild.org/v2.5/docs/existing-repositories#3-set-up-tests

Is there a way to opt-in to anonymous-telemetry only on my machine? (like in

~/.pants.toml

)? I'm interested in providing it on my machine and in CI, but I don't want to turn it on by default on other people's machines (when they check out the open source repo).

I'm trying to implement scripts similar to what's in

build-support/bin

in

pantsbuild/pants

, but having trouble getting it to run. I have the directory in

[source].root_patterns

, and a

pex_binary

in

build-support/bin

but I feel I'm missing something because I get this error:

λ › ./pants run build-support/bin/changelog.py                                                                                                                                      odl/ol-django nl/pants-t2
Traceback (most recent call last):
  File "/home/nathan/odl/ol-django/.pants.d/tmpstd0qzkb/changelog.pex/.bootstrap/pex/pex.py", line 489, in execute
  File "/home/nathan/odl/ol-django/.pants.d/tmpstd0qzkb/changelog.pex/.bootstrap/pex/pex.py", line 406, in _wrap_coverage
  File "/home/nathan/odl/ol-django/.pants.d/tmpstd0qzkb/changelog.pex/.bootstrap/pex/pex.py", line 437, in _wrap_profiling
  File "/home/nathan/odl/ol-django/.pants.d/tmpstd0qzkb/changelog.pex/.bootstrap/pex/pex.py", line 545, in _execute
  File "/home/nathan/odl/ol-django/.pants.d/tmpstd0qzkb/changelog.pex/.bootstrap/pex/pex.py", line 672, in execute_entry
  File "/home/nathan/odl/ol-django/.pants.d/tmpstd0qzkb/changelog.pex/.bootstrap/pex/pex.py", line 684, in execute_module
  File "/home/nathan/.pyenv/versions/3.6.10/lib/python3.6/runpy.py", line 201, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/home/nathan/.pyenv/versions/3.6.10/lib/python3.6/runpy.py", line 136, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named changelog

Does the corresponding BUILD file have a python_library target in addition to the pex_binary target(s)? It's unfortunately needed for now for pants internal book-keeping: https://github.com/pantsbuild/pants/blob/main/build-support/bin/BUILD#L6

it did not, that did the trick, thanks!

Great. If you look closely at the two methods for specifying a pex_binary entry_point described here: https://www.pantsbuild.org/docs/python-package-goal#approach-1-a-file-name https://www.pantsbuild.org/docs/python-package-goal#approach-2-explicit-entry_point You might miss the

python_library()

at the top of the BUILD file. As you found out - that's critical. It's also easy to miss because its not clear why you should need to have it. You might be able to save pain going forward (I still often forget to add a python_library) by using

./pants tailor

. More on that here: https://www.pantsbuild.org/docs/troubleshooting#import-errors-and-missing-dependencies

It makes sense in hindsight, I would've needed it anyway for it to discover sibling modules I'm importing

Hi, is there a best practice on how to do "cross compiling" with 2 different sets of dependencies? I have a BUILD that produces a fatjar with a whole tree of dependencies under it. At various parts of the tree it needs

libx=1.1

. I'd like to build the fatjar once with

libx=1.1

and once with

libx=1.2

. I can think of 2 ways to do it: a) At the top fatjar, exclude all libx dependencies, and then include the version I want b) At every point in the tree, pass some flag (env variable?) so that it can switch between 1.1 and 1.2 And of course, I'd like to avoid the solution that requires a copy of all the

BUILD

files. Thanks for any pointers.

Hi Pants experts, I am trying to build airflow dags into pex files (see this repo). The problem is I am unable to add dependencies to the contraint.txt when I run "build-support/generate_constraints.sh". Can anyone please show me what is wrong? Thanks!

https://twitter.com/pantsbuild/status/1392568104752861185 It's PyCon US Time, and Pants Build has a major presence this year! Find out about our talks and demos here: https://t.co/P7ltJ0yyAs Hope to see you at the conference (albeit virtually...)! Twitter

Hello! Has anyone seen this error in their repository?

NailgunClientException: Nailgun client error: "Nailgun client error: Client exited before the server\'s result could be returned."

We're trying really hard to track it down, and it's been difficult to reproduce.

I've got PyYAML in both requirements and constraints files. but it's not getting inferred in a file with

import yaml

$ ./pants dependencies st2common/st2common/constants/meta.py
st2common/st2common/__init__.py
st2common/st2common/conf/base.logging.conf:base_logging_conf
st2common/st2common/constants/__init__.py

Why isn't it picking up the yaml dep? https://github.com/st2sandbox/st2/blob/pants/st2common/st2common/constants/meta.py https://github.com/st2sandbox/st2/blob/pants/requirements-pants.txt#L40 https://github.com/st2sandbox/st2/blob/pants/constraints-pants.txt#L89 https://github.com/pantsbuild/pants/blob/2.5.x/src/python/pants/backend/python/dependency_inference/default_module_mapping.py#L19

Should setuptools always be present? Because it doesn't seem to be. I'm getting some import errors for

pkg_resources

which is in

setuptools

a dep,

flex

does not declare a dep on

setuptools

(they think its std lib): https://github.com/pipermerriam/flex/blob/master/flex/__init__.py#L6 What are my options for getting pants to include setuptools in this case?

Quick question. Pants v1 was supporting other languages besides Python as far as I can tell. Pants v2, does not. Is there a plan to support other languages in the future, or is this something we need to do on our own through the Plugin System?

OK. I'm going to rephrase a question I asked earlier. I want pants to be the primary entry point for development, replacing the Makefile & supporting scripts. One responsibility of the Makefile (and a couple of scripts) is setting up a local development environment with services like mongo, rabbitmq, and redis. How can I do setup like that from within pants? Would I need to write a pants plugin? Something that makes sure the services are running when running tests? How would service requirements (mongo, rmq, redis, etc) interact with remote execution (REAPI)? (which is something I'm interested in using eventually)