Hi guys. Wondering if anybody else has been able to run

shell_plus --notebook

for their Django app with pants. We are having a lot of issues with dependencies doing it.

Did I write a fair description of

[anonymous-telemetry]

in https://github.com/StackStorm/st2/pull/5737 ? If I can clarify that before I mark it ready for review (by the StackStorm maintainers), then that would be good.

Hi @bitter-ability-32190. I tried you solution using a

file

target and declaring a dependency to have the

.env

file in the

run

sandbox, and it worked. I have a problem now with CI/CD. When I try to run Pants in GitHub Actions (so without an

.env

file), it fails because there isn't an

.env

file available (the environment variables are already set up by GitHub Action). Do you have a solution for that?

Hi guys. Currently trying to run pants with a docker container for local development. I have restartable set to true, but when I make changes, the entire docker rebuilds, then fails to bind bind to the port I had set because it's failing to kill the old container. Anybody have a fix for this? Or is this a bug?

Can you just

touch

one?

who made the

p

pants helper bash plugin/script ? and where is it? Cant find it >_>

after some searching in the repo i was able to use

run_in_sandbox=False

in my pex_binary

pex_binary(
    name="manage_dev",
    entry_point="manage.py",
    run_in_sandbox=False,
    restartable=True,
    dependencies=[":deps", ":reqs_dev"],
)

but its not documented i believe? https://www.pantsbuild.org/docs/reference-pex_binary

Not sure what changed on our end yet, but up to as recently as yesterday morning generate-lockfiles worked just fine, but now no matter what I do I get a KeyError for any input requirement that is pinned in some way, e.g., ipywidgets<8.0 — stacktrace below. I've tried nuking the cache, switching between 13 and 14 pants versions, and no luck. We can still build pex_binarys from the current lock file. Just wondering if anything immediately comes to mind that would cause this behavior.

pex: Resolving interpreters :: Indexing downloads

Traceback (most recent call last):

File "/d/d1/user/taymarti/.cache/pants/named_caches/pex_root/installed_wheels/dd24a2333561b849fbebe03fe7319585ddb9a8ad512920b6bb2d9249a810cb71/pex-2.1.104-py2.py3-none-any.whl/pex/result.py", line 105, in catch

return func(*args, **kwargs)

File "/d/d1/user/taymarti/.cache/pants/named_caches/pex_root/installed_wheels/dd24a2333561b849fbebe03fe7319585ddb9a8ad512920b6bb2d9249a810cb71/pex-2.1.104-py2.py3-none-any.whl/pex/cli/command.py", line 84, in run

return subcommand_func(self)

File "/d/d1/user/taymarti/.cache/pants/named_caches/pex_root/installed_wheels/dd24a2333561b849fbebe03fe7319585ddb9a8ad512920b6bb2d9249a810cb71/pex-2.1.104-py2.py3-none-any.whl/pex/cli/commands/lock.py", line 432, in _create

create(

File "/d/d1/user/taymarti/.cache/pants/named_caches/pex_root/installed_wheels/dd24a2333561b849fbebe03fe7319585ddb9a8ad512920b6bb2d9249a810cb71/pex-2.1.104-py2.py3-none-any.whl/pex/resolve/lockfile/create.py", line 311, in create

create_lock_download_manager = CreateLockDownloadManager.create(

File "/d/d1/user/taymarti/.cache/pants/named_caches/pex_root/installed_wheels/dd24a2333561b849fbebe03fe7319585ddb9a8ad512920b6bb2d9249a810cb71/pex-2.1.104-py2.py3-none-any.whl/pex/resolve/lockfile/create.py", line 83, in create

artifact = file_artifacts_by_filename.get(f) or source_artifacts_by_pin[pin]

KeyError: Pin(project_name=ProjectName(raw='ipywidgets', normalized='ipywidgets'), version=Version(raw='7.7.2', normalized='7.7.2'))

Pin(project_name=ProjectName(raw='ipywidgets', normalized='ipywidgets'), version=Version(raw='7.7.2', normalized='7.7.2'))

I'm curious if there's a way to speed up lockfile generation or incrementally modify the lockfile, e.g. add a single new dependency or upgrade a single dependency version. It currently takes about 2 minutes just for my subproject to generate the lockfile, even when no changes have been made to any

python_requirements

. Setting up a virtual environment with our existing tooling takes no more than 10 seconds after dependencies have been downloaded.

After making some proof of concept for a self-contained subproject I'm trying to run

./pants check subproject/some/deep/test/directory::

, but the tool complains that a test directory in a completely different subproject isn't covered by the source roots. Is there a way to diagnose why Pants is complaining about this? I tried

./pants dependencies subproject::

and it has the same problem.

Ruh roh:

Error storing fingerprints ["000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20"]: MDB_MAP_FULL: Environment mapsize limit reached

Hi all, I'm trying to convert our repository into a mono-repo by splitting code into multiple python packages. To this end, I've defined two packages under

packages/pkgA

and

packages/pkgB

with their individual

pyproject.toml

files, using poetry. However, while editing the dependencies, I just noticed that the code under

pkgB

imports

numpy

and

pants dependencies

shows

packages/pkgA:poetry#numpy

. This means that the tests of

pkgB

run successfully, even though a dependency is missing, because it is present in another package within the repo. Is it possible to disable this behavior and have them only look up

packages/pkgB:poetry#*

or fail if not found? PS. Currently on pants 2.12 if it matters and not using lockfiles (yet)

i have managed to get my tailwind compiler "working" ->

./pants export-codegen

is exporting a css file. but somehow i'm to stupid to get that into my python project. i have a dependency defined in my python_sources pointing to my custom tailwind_sources rule i am surely missing something, but i dont get it after reading the pants source

Having some periodic trouble initializing any lockfile that contains backports.cached_property=1.02. It was working for a bit and now it doesn't work either for the default PyPi index or our internal mirror, and neither on our VPN or on regular internet. Easy repro attached

git clone master.bundle
cd master
./pants generate-lockfiles --resolve=test

Is there a way to disable logging to

.pants.d/run-tracker/*

directory? Or just disable logging to files altogether? For context, we are running pantsbuild along with another tool(skaffold) which results in pants goals being called in a loop, sometimes many times a second. This results in a proliferation of the

.pants.d/run-tracker/pants_run_YYYY_MM_DD_MM_SS_SHA

folders which are not really necessary. Since the

.pants.d

folder is required to be in the repository this is causing issues with some IDEs where file watching is not as configurable.

I'm getting an import error when I try to run some tests that doesn't make sense to me. My test is failing on

import fastapi

with

ModuleNotFoundError: No module named 'requests'

, which is a dependency of FastAPI. I tried explicitly adding

requests

to my

requirements.txt

and rebuilding the lock file, but it didn't clear the error. This seems funny to me - the code I am trying to test (using FastAPI) builds and runs fine, it's only the test itself that's failing on this import. Any advice? Some more details in 🧵

any version of

scala_from_source

for a github branch/ref? or a nightly version of pants?

I was hoping to get some help with this issue. The pex generated in my code doesn't respect the proper version of GDAL that's installed in the container. I've tried pinning GDAL 3.3.2 in my global build file, clearing any and all caches I can, however the pex still tries using GDAL 2.4.0 https://pantsbuild.slack.com/archives/C046T6T9U/p1663622079109119

Hi folks! This is my occasional reminder that github stars do help promote the project to new orgs and users, and so it would super appreciated if you could give us some starry love at https://github.com/pantsbuild/pants (and encourage your org’s internal Pants users to do the same, if you’re really feeling inspired) This benefits the project, which in turn benefits us all. So thanks!

Hello all, I'm writing a linter plugin for pants to run the linter golangci-lint (which is a popular meta-linter for go). I have pants running golangci-lint, however pants is splitting my project into two sandboxes. This causes golangci-lint to fail because neither sandbox is fully complete. I'm getting similar behavior with the go-vet linter, so I'm assuming its something about my project. When I run go-vet via pants, it fails with:

config/config.go:26:2: module lookup disabled by GOPROXY=off
cmd/stentor/stentor.go:33:2: <http://github.com/davecgh/go-spew@v1.1.1|github.com/davecgh/go-spew@v1.1.1>: module lookup disabled by GOPROXY=off
cmd/stentor/stentor.go:34:2: <http://github.com/davecgh/go-spew@v1.1.1|github.com/davecgh/go-spew@v1.1.1>: module lookup disabled by GOPROXY=off
cmd/stentor/stentor.go:35:2: <http://github.com/davecgh/go-spew@v1.1.1|github.com/davecgh/go-spew@v1.1.1>: module lookup disabled by GOPROXY=off

However, if I run

go vet ./...

in the sandbox, it fails with a clearer error:

cmd/stentor/stentor.go:33:2: no required module provides package <http://github.com/wfscheper/stentor/internal/templates;|github.com/wfscheper/stentor/internal/templates;> to add it:
        go get <http://github.com/wfscheper/stentor/internal/templates|github.com/wfscheper/stentor/internal/templates>
cmd/stentor/stentor.go:34:2: no required module provides package <http://github.com/wfscheper/stentor/newsfile;|github.com/wfscheper/stentor/newsfile;> to add it:
        go get <http://github.com/wfscheper/stentor/newsfile|github.com/wfscheper/stentor/newsfile>
cmd/stentor/stentor.go:35:2: no required module provides package <http://github.com/wfscheper/stentor/release;|github.com/wfscheper/stentor/release;> to add it:
        go get <http://github.com/wfscheper/stentor/release|github.com/wfscheper/stentor/release>

Pants is aware of these dependencies:

./pants dependencies cmd/stentor/stentor.go
22:19:24.75 [INFO] Initialization options changed: reinitializing scheduler...
22:19:25.18 [INFO] Scheduler initialized.
//:root#<http://github.com/stretchr/testify/assert|github.com/stretchr/testify/assert>
//:root#<http://github.com/stretchr/testify/require|github.com/stretchr/testify/require>
... # Trimmed out testdata files
config:config
fragment:fragment
internal/templates:templates
internal/test:test
newsfile:newsfile
release:release

When generating a lockfile, I'm getting an error because pants seems to add a requirement

setuptools<63.0.0,>=62.3.2

even though I haven't specified any version. This clashes with a package (

numba>=0.56.1

) that requires

setuptools<60

for some reason. Is there a way around this? I tried setting

PANTS_SETUPTOOLS_VERSION="setuptools<60"

but it had no effect. This is the error I'm getting:

$ ./pants --no-watch-filesystem --no-pantsd generate-lockfiles --resolve=python-default
14:31:15.17 [INFO] Completed: Generate lockfile for python-default
14:31:15.17 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:

stderr:
ERROR: Cannot install setuptools<60 and setuptools<63.0.0 and >=62.3.2 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit <https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies>
... very long pex command...

 The conflict is caused by:
     The user requested setuptools<63.0.0 and >=62.3.2
     numba 0.56.2 depends on setuptools<60

Any ideas how to solve this? Edit: This is on pants 2.12.0

Pants TIP of the day: When you’re working with “magic” libraries, such as Django, there are times where you have 3rd party dependencies that are not inferred as they don’t materialize in your 1st party code. At times there are hints to use, though, as shown in this little example. Say we have a module, that uses the postgresql feature:

import django.contrib.postgres

and that module in turn uses

psycopg2

but it isn’t inferred nor provided as a dependency of Django itself as it’s optional. You can use the module mapping feature in Pants to your advantage in this case, with a snippet like this:

python_requirements(
    module_mapping={
        "psycopg2": [
            "psycopg2",
            # This is for dependency inference only, when we see an import for
            # `django.contrib.postgres` we should infer a dependency on `psycopg2`.
            "django.contrib.postgres",
        ],
    },
...

Pants will properly infer the dependency for you correctly. 🎉

Hello… I’m having a small issue where my static files aren’t available(loading

404

) when I run a shell script (

shell_sources

) that among other things executes

collectstatic

This is only happening when I run the docker version. Here’s how the

Dockerfile

looks like

FROM python:3.10-slim

### Arg
ARG DEBIAN_FRONTEND=noninteractive

### Env
ENV APP_HOST=.
ENV APP_DOCKER=/app
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

COPY twoops_tracker.py/main.pex /bin/dj

# Expose server port
EXPOSE 8000

### Volumes
WORKDIR ${APP_DOCKER}
RUN mkdir -p media staticfiles logs

COPY twoops_tracker/sh/*.sh /
RUN chmod +x /cmd.sh

### Run app
CMD ["/cmd.sh"]

I’ve also noticed that the files are being saved under

/root/.pex/user_code/a517e67b58e274b35127ae63804115e6156bfe3d/staticfiles

which I think is the reason why they can’t be found. Any ideas on how I can solve this?

In pants 2.13, is there a way to exclude

__init__.py

files in the pants config instead of the BUILD files? Migrating from pants 1.30 and it seems like 2.13 wants to run every file through pytest, however we have plenty of

__init__.py

and

common.py

files which we don't want to make pytest attempt to execute. Was hoping to have a global config rather than have every BUILD file have a statement in the sources that tells it to ignore the names I mentioned

Hello. I'm still on pants 2.12 and I building some docker images with pants. Some of the images depend on a base image that is also built at the same time. The "leaf" images have a dependency on the target building the base image in their BUILD file but it seems that pants is building the docker images out of order and so it fails by saying that my base image is not found. Does pants manage this kind of dependency? It looks like it tries to build everything in parallel but I can't confirm for sure.

When building a docker image with a dynamic tag, what's the best way to capture this for use by other processes? For context, we want to have 'content-addressed' docker images (managed by pants, rather than docker's caching) with a tag like

image_tags=["{pants.hash}"]

and stick this into a CloudFormation template to have the image running in AWS as a Lambda or Fargate service or whatever. The only approaches I can determine are not so good: 1. capture it out of the logging of a

./pants package ...

call: our system is then doing automatic processing of human facing messaging 2. looking at the most recent row of `docker images --format '{{.Tag}}' $my_repository`: we have to write the repository out in yet another place, and, with caching/content-addressing, there's no guarantee that the most recently created image is the one that's been created as part of the current build 3. (there doesn't appear to be anything recorded in

dist/

, despite this being a

package

call; and something like

peek

gives me seems to give me

image_tags

before formatting)

I’m experiencing an indefinite `Filesystem changed during run: retrying

Test

in 500ms...` loop while trying to run multiple tests. I was AFK for about 20 minutes and came back to this (thread). Any experience here or any good advice?

Hi, I am facing with a very slow

run

goal completion time. Right now we have a python project with ~50 dependencies and 35-40k LOC. When I run

./pants --level=debug --pex-verbosity=9 run /path/to/folder:binary

it takes a whopping 70-80 secs to complete. And when I add a print statement to the entry file it takes the same time 70-80 seconds. I tried using

execution_mode=venv

and

pex.venv_use_symlinks=true

options but they didn’t help. Here is the timing details:

pex: Building pex: 8413.4ms
pex:   Laying out PEX zipfile local_dists.pex: 0.1ms
pex:   Searching dependency cache: /Users/serhat/.cache/pants/named_caches/pex_root/unzipped_pexes/53b03e2e28c1a38ab2a682f3a8d615cd2ba49035/.deps: 0.0ms
pex:   Resolving distributions (...): 8325.8ms
pex:     Parsing lock third_party/requirements_macos_arm64.lock: 335.4ms
pex:     Resolving requirements from lock file third_party/requirements_macos_arm64.lock: 7974.6ms
pex:       Parsing requirements: 7.3ms
pex:       Resolving urls to fetch for 48 requirements from lock third_party/requirements_macos_arm64.lock: 113.1ms
pex:       Hashing pex: 161.1ms
pex:       Isolating pex: 0.1ms
pex:       Downloading 131 distributions to satisfy 48 requirements: 43.7ms
pex:       Categorizing 131 downloaded artifacts: 1.0ms
pex:       Building 27 artifacts and installing 393: 7641.5ms
pex:         Building distributions for:
...
pex:         Installing 131 distributions: 7594.7ms
pex: Vendoring attr from VendorSpec(key='attrs', requirement='git+<https://github.com/python-attrs/attrs@947bfb542104209a587280701d8cb389c813459d#egg=attrs>', rewrite=True, constrain=False, constraints=None) @ /Users/serhat/.cache/pants/named_caches/pex_root/installed_wheels/6a3c9545f9bc666f039874243789bf4ba5cc4fbde5a945eca40a651de360568b/pex-2.1.103-py2.py3-none-any.whl/pex/vendor/_vendored/attrs
pex: Vendoring packaging from VendorSpec(key='packaging', requirement='packaging==20.9', rewrite=True, constrain=True, constraints=('pyparsing<3',)) @ /Users/serhat/.cache/pants/named_caches/pex_root/installed_wheels/6a3c9545f9bc666f039874243789bf4ba5cc4fbde5a945eca40a651de360568b/pex-2.1.103-py2.py3-none-any.whl/pex/vendor/_vendored/packaging
pex: Vendoring pyparsing from VendorSpec(key='packaging', requirement='packaging==20.9', rewrite=True, constrain=True, constraints=('pyparsing<3',)) @ /Users/serhat/.cache/pants/named_caches/pex_root/installed_wheels/6a3c9545f9bc666f039874243789bf4ba5cc4fbde5a945eca40a651de360568b/pex-2.1.103-py2.py3-none-any.whl/pex/vendor/_vendored/packaging
pex: Vendoring pkg_resources from PreparedGit(key='setuptools', requirement='git+<https://github.com/pantsbuild/setuptools@3acb925dd708430aeaf197ea53ac8a752f7c1863#egg=setuptools>', rewrite=True, constrain=False, constraints=None) @ /Users/serhat/.cache/pants/named_caches/pex_root/installed_wheels/6a3c9545f9bc666f039874243789bf4ba5cc4fbde5a945eca40a651de360568b/pex-2.1.103-py2.py3-none-any.whl/pex/vendor/_vendored/setuptools
pex: Vendoring setuptools from PreparedGit(key='setuptools', requirement='git+<https://github.com/pantsbuild/setuptools@3acb925dd708430aeaf197ea53ac8a752f7c1863#egg=setuptools>', rewrite=True, constrain=False, constraints=None) @ /Users/serhat/.cache/pants/named_caches/pex_root/installed_wheels/6a3c9545f9bc666f039874243789bf4ba5cc4fbde5a945eca40a651de360568b/pex-2.1.103-py2.py3-none-any.whl/pex/vendor/_vendored/setuptools
pex: Vendoring wheel from VendorSpec(key='wheel', requirement='wheel==0.37.1', rewrite=False, constrain=True, constraints=None) @ /Users/serhat/.cache/pants/named_caches/pex_root/installed_wheels/6a3c9545f9bc666f039874243789bf4ba5cc4fbde5a945eca40a651de360568b/pex-2.1.103-py2.py3-none-any.whl/pex/vendor/_vendored/wheel
Saving PEX file to <http://src.python.app/app.pex|src.python.app/app.pex>
pex: Zipping PEX file.
pex: Zipping PEX file.: 53163.6ms
16:22:24.36 [DEBUG] Completed: pants.backend.python.util_rules.pex.build_pex
16:22:24.36 [DEBUG] Dependency @rule(pants.backend.python.util_rules.pex.create_pex()) of Some("@rule(pants.backend.python.goals.package_pex_binary.package_pex_binary(<http://src.python.app/app.pex|src.python.app/app.pex>))") changed.
16:22:24.36 [DEBUG] Completed: pants.backend.python.goals.package_pex_binary.package_pex_binary
16:22:24.36 [DEBUG] Dependency @rule(pants.backend.python.goals.package_pex_binary.package_pex_binary(<http://src.python.app/app.pex|src.python.app/app.pex>)) of Some("@rule(pants.backend.python.goals.run_pex_binary.create_pex_binary_run_request(<http://src.python.app/app.pex|src.python.app/app.pex>))") changed.
16:22:24.37 [DEBUG] Completed: pants.backend.python.goals.run_pex_binary.create_pex_binary_run_request
16:22:24.76 [DEBUG] Completed: setup_sandbox
...
16:22:37.42 [DEBUG] Completed: `run` goal
16:22:37.42 [DEBUG] computed 1 nodes in 78.639370 seconds. there are 12360 total nodes.

Is this normal? It’s my first time working and trying to adopt pants in our monorepo. We use bazel right now.

This feels like a very stupid question but I can’t seem to find the answer: I have a

pex

file that I want to execute by simply running

./my_package.pex

, but I cannot figure out how to pass arguments to the script that’s inside the package. Just adding flags doesn’t seem to get parsed, and passing them via

--

doesn’t work either.

Hey pants team, is there a way to investigate what is pulling in a specific wheel file? I have an odd issue I can't seem to track down where a pex I have build (using pants 1.30 and pants 2.13) references a completely different version of a 3rd party module than I have referenced in the BUILD file.