Really rapid firing here aren't I. This is a silly question: Is there any way to explicitly allow a dependency conflict. I'm trying to use a package that wants

protobuf<3.18

. There were some important security vulnerabilities fixed since that we have to restrict to

protobuf>3.19

in our repo. Result is there is no way for me to add the package to my project. The risk of not being able to resolve dependencies is that the project may not work, but I'm fairly confident it will with some tickets I've seen. So, I'm wondering if it's possible to ignore this dependency mismatch while I wait for an upstream fix? The only alternative I can think is to fork the dependency and bump it myself, but that seems like a large undertaking.

Hi team, I am facing the following error while I run tests that use jupyter notebook in pants. The test looks like

@testbook(PATH_NOTEBOOKS / "prep.ipynb", execute=True)
def test_prep_notebook(tb: TestbookNotebookClient):
    print_notebook_output(tb)

The error is like

/.cache/pants/named_caches/pex_root/venvs/s/86f08450/venv/lib/python3.9/site-packages/jupyter_client/client.py:202: in _async_wait_for_ready
    raise RuntimeError("Kernel died before replying to kernel_info")
E   RuntimeError: Kernel died before replying to kernel_info

How to solve this issue? Thanks you!

Another quick question: what is right syntax for pants_ignore.remove? I know we can do pants_ignore.add.

Would it be useful for more people than me to have an archive format of "directory" that would just copy files without packing them in a tar or zip?

Given the following BUILD file in my repo

./nectl/neuroedge_cli/BUILD

python_sources(name = "neuroedge_cli")

pex_binary(
    name = "nectl",
    dependencies = [":neuroedge_cli"],
    entry_point = "./nectl.py",
)

If I run

./pants run nectl/neuroedge_cli/nectl.py

Everything is fine. If I instead run

./pants run nectl/neuroedge_cli:nectl

It fails with:

12:58:13.40 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building nectl.neuroedge_cli/nectl.pex with 10 requirements: click==8.1.3, dvg-ringbuffer==1.0.3, keyring==23.5.1, numba==0.56.0, numpy==1.22.4, pandas~=1.4, requests==2.25.1, rich==12.4.1, scipy==1.8.1, tables==3.7.0' failed with exit code 1.
stdout:

stderr:
pid 28706 -> /Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/e6831f8bef1e0125d178dbca2c603370d4eeae8a/bin/python -sE /Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/e6831f8bef1e0125d178dbca2c603370d4eeae8a/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/russellzarse/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/_d/zt8y2x055597l63bhdztp82c0000gn/T/pants-sandbox-8W6RvT/.tmp/pex-pip-log.ooax9v3d/pip.log download --dest /Users/russellzarse/.cache/pants/named_caches/pex_root/downloads/resolver_download.hetwm61n/usr.local.Cellar.python@3.11.3.11.0.Frameworks.Python.framework.Versions.3.11.bin.python3.11 click==8.1.3 dvg-ringbuffer==1.0.3 keyring==23.5.1 numba==0.56.0 numpy==1.22.4 pandas~=1.4 requests==2.25.1 rich==12.4.1 scipy==1.8.1 tables==3.7.0 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
WARNING: Discarding <https://files.pythonhosted.org/packages/b3/23/fd8e7aa70f6c0b41c99de6aae7afc6850ebac2477687e68c6529bfaa41ba/numba-0.56.0.tar.gz#sha256=87a647dd4b8fce389869ff71f117732de9a519fe07663d9a02d75724eb8e244d> (from <https://pypi.org/simple/numba/>) (requires-python:>=3.7). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement numba==0.56.0
ERROR: No matching distribution found for numba==0.56.0

If I add

interpreter_constraints

to the

pex_binary

target

pex_binary(
    name = "nectl",
    interpreter_constraints=['CPython==3.8.*'],
    dependencies = [":neuroedge_cli"],
    entry_point = "./nectl.py",
)

run succeed. However...

pex_binary(
    name = "nectl",
    interpreter_constraints=['CPython>=3.8,<4'],
    dependencies = [":neuroedge_cli"],
    entry_point = "./nectl.py",
)

fails in the same way Why does the pex_binary require explicit constraints? Shouldn't the

CPython>=3.8,<4

configuration work?

Hey there! I have made good progress, but one thing still not working, I have 3 Libraries in Python ( I can run all test/lint/fmt for all libraries) and now I am trying to add a Project that use those 3 Libraries (local dependencies), All projects use Poetry but when I run the tests for the Project, it won't detect my dependencies. my

project/BUILD

looks like this

python_sources(
    name="project",
    dependencies = [
        "src/libraries/lib1:poetry",
        "src/libraries/lib2:poetry",
        "src/libraries/lib3:poetry",
    ]
)

poetry_requirements(
    name="poetry",
)

and all lib looks like this

poetry_requirements(
    name="poetry",
    module_mapping={
        [...] # multiple modules mapping 
    },
    overrides={
        [...] # multiple overrides
    }
)

it looks like it getting the local dependencies since its throwing errors from inside the library

src/libraries/lib1/lib1/v2/enums/base_format.py:5: in <module>
    import yaml
E   ModuleNotFoundError: No module named 'yaml'

but if I run the tests of

lib1

everything is ok. Local Dependencies are already declared as Editable in the

pyproject.toml

Hello again - another pex question. Is there an easy way to create a venv for a pex that was built without

--include-tools

or

--venv

? I found a roundabout way to do this, is there something simpler?

pex --include-tools -o just-tools.pex
PEX_TOOLS=1 PEX_PATH=the-pex-without-tools.pex ./just-tools.pex venv my-venv

While can build our user's newer pexes with

--include-tools

, we also need to support their older pexes which were not built with tools.

I have the

[pylint].source_plugins

in a separate resolve from the rest of the code. That has been working very nicely. When I run pylint, the venv includes pylint, the source_plugins, and the code that pylint is inspecting. So far so good. But, now I want to test code in my source_plugins directory, and 💥. For my “fixtures” I have the test write a dummy python file from a string to a temporary directory. That dummy file imports a module that is in a different resolve. If I were running pylint, this module would be part of the code that pylint inspects, which works just fine. For testing, how do I make that module available to import during the test?

Common problem my team has: Google/DuckDuckGo ranks the v1 documentation higher than the v2 documentation and keeps confusing my devs.

I can't find any page explaining what the pex tools are and do, is there any?

Question about building a pex into a Docker image. I need to reference some files in my repo - the code does things like

shutil.copytree(src/services/app/static, etc)

. This works fine when I run the pex target, but once I build a Docker image from it I get

FileNotFoundError: [Errno 2] No such file or directory: 'src/services/app/static'

. I tried adding the directory as both a

files

and

resources

and having my docker target depend on it, but that didn't help. What's the magic I'm looking for here? Am I not referencing it right in the Docker image, or are those files not going to be there at all without some extra work? For reference, my dockerfile itself is simply

FROM python:3.8
ENTRYPOINT ["/bin/app.pex"]
COPY src.services.app.scripts/app.pex /bin

Is it possible to use the

vcs_version

target as a tag for the `docker_image`target? Do I need to write a small plugin?

Recenty, Github Action CI got this bootstrap error. I have seen something has been changed(link) in example-django ropo. Do I need to change something to fix this?

Run mkdir .tmp
  mkdir .tmp
  ./pants --no-verify-config version
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.9.16/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.16/x64/lib
  
./pants: line 400: /home/runner/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/bin/python: No such file or directory
Error: Process completed with exit code 127.

I have tried package docker but got an error after pex build. Searching but no result. Is that something related local docker configuration??

12:21:12.39 [INFO] Completed: Building docker image docker:latest
12:21:12.40 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building docker image docker:latest' failed with exit code 1.
stdout:

stderr:
#1 [internal] load build definition from Dockerfile.superman
#1 sha256:d00f80a10014e5eb26d752fef9acfd03ed0d23487d4e93688ad0a6753a8a42cf
#1 transferring dockerfile: 144B 0.0s done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:c686b491462bcaba2ca2d35328c1948309e89ce47a5300a88f8b18edf0a79466
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for <http://docker.io/library/python:3.9|docker.io/library/python:3.9>
#3 sha256:5ceb849adf4ca2eed629b09d8add870d381c45f6f40f947be68f6595c97a8af8
Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER, $HOME set in environment
[2022-12-16T03:21:12.378018000Z][docker-credential-desktop][F] get system info: exec: "sw_vers": executable file not found in $PATH
[goroutine 1 [running, locked to thread]:
[common/pkg/system.init.0()
[       common/pkg/system/os_info.go:32 +0x29d
#3 ERROR: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: ``
------
 > [internal] load metadata for <http://docker.io/library/python:3.9|docker.io/library/python:3.9>:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: ``



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

Hi, I have just upgraded from 2.10 to 2.14 pants. We used to have empty BUILD files earlier. Somewhat like

python_sources()

python_tests(
name="tests",
)

And pants used to figure out all the dependencies. But now after the upgrade, all the test cases with above BUILD files are failing since pants is not able to figure out the dependencies. Is there a setting I am missing or this is no longer supported?

On Python3.9, alpine 3.15 I get:

ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.14.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0)
ERROR: No matching distribution found for pantsbuild.pants==2.14.0

Hello all - apologies for what feels like a question I should have been able to answer myself - I’d like to exclude paths from

./pants check ::

- specifically using mypy. It seems that Pants runs mypy per-file, which as far as I understand ignores any

exclude

arg passed to mypy. I don’t want to typecheck my tests - is that something I can do?

Hello, is there a way to configure the buf linter with a buf.yaml?

Does the lack of aarch64 linux support mean that I can’t use the docker target from a M1 Mac?

after running an

export-codegen

today that generated a ton of small files, I started getting “pantsd OOMKilled” errors on every following run, even on other commands. running with

--no-pantsd

now shows:

/Users/runner/.cargo/git/checkouts/lmdb-rs-369bfd26153a2575/6ae7a55/lmdb-sys/lmdb/libraries/liblmdb/:2126: Assertion 'rc == 0' failed in mdb_page_dirty()
Abort trap: 6

has anyone hit this before & know what it means?

This morning's podcast interview with @happy-kitchen-89482 is up: https://anchor.fm/happypathprogramming/episodes/72-Pants-Makes-Developers-Happier--More-Productive-with-Benjy-Weinberger-e1sc1uj

Hi folks. Sorry for the (probably) repeat question, but couldn't find searching: what's the latest on C/C++ support? I see some cc_sources in master, but it doesn't appear to be enabled. Any examples out there? I'm looking to have a Python+C++ through ctypes setup.

I'm finding

pantsd

is restarting often, presumably because it's hitting the memory limit. We can just increase the memory limit, but is there a way to understand more about what's going on (e.g. in case we have some simple misconfiguration)? I tried

./pants --stats-memory-summary ...

but that only reports 10MB (summing all rows), very different to htop reporting 400MB-500MB resident.

I'm having some issues with python interpreter constraints. I had one default interpreter constraint that I'm using for my primary resolve and all the tools. But, I also have a separate

pants-plugins

resolve that matches pants' interpreter constraints. When I started trying to run some tests, I ran into some conflicting constraints (InvalidLockfileError), so I tried making the default constraints be a superset of all the resolves. https://github.com/StackStorm/st2/pull/5847/commits/9f26453cb0ff86ff6007a8acc093673431916efc But now I'm getting different constraints conflict (InvalidLockfileError) issues: https://github.com/StackStorm/st2/actions/runs/3718138874/jobs/6306123192#step:5:32 What am I missing? (I'm on pants 2.14.0)

Hi! I'm trying to run Pants in a Replit.com Linux VM. When running the one_step_setup.sh I got the following error. Any ideas? Thanks! "An error occurred during configuration: option use-feature: invalid choice: 'content-addressable-pool' (choose from '2020-resolver', 'fast-deps', 'in-tree-build')"

Hi! 👋 Is there way to specify interpreter constraints when

./pants generate-lockfiles

for specific requirements.txt? need some guidance on how to setup codebases requiring multiple python versions. We have 2 big codebases. 1.

py3.7

used in airflow, another

py3.10

used in dockerized apps. To support these, I went with 2 requirements.txt.

3rdparty/airflow/requirements.txt

and

3rdparty/python/requirements.txt

. Added

python_sources(resolve="airflow-default", interpreter_constraints=["CPython>=3.7.10, <3.8"],)

to all BUILD files for airflow codebases. Also changed my

pants.toml

to this:

[python]
interpreter_constraints = ["CPython>=3.7.10, <3.8", "==3.10.*"]

My local python is version

3.10.8

The issue is

./pants generate-lockfiles

when resolving packages for airflow seems to be looking for packages which support 3.10.8. Which causes resolution issues.

I briefly had a conversation with John about this, but does anyone have concrete examples on how to package docker images within Gitlab CI? It seems to me that Pants is tightly coupled to the

docker

binary. I can't use dind since it needs privileged access. I usually use kaniko, but that does not have the same CLI as docker. Tried Podman, but it complains about

nsenter

when run from pants.

Hi. Is it possible to use different --extra-index-url for different resolves. I'd like to use different versions of pytorch in different apps, but the difference between them is sometimes only in the index url. So for example I'd like to use pytorch from https://download.pytorch.org/whl/cpu in resolve_a and https://download.pytorch.org/whl/cu116 in resolve_b

Hope everybody is having a good monday so far! Small question, what would be the best approach to include multiple folder with static files like, I have see the

resources

and the

files

targets and wondering how I can define correctly this dependencies. I have already done it for a single file with

resource()

and add it to the

python_sources

dependencies but for folder that I want to include everything. Thanks in advance 🙂

Hey all, Is there anyway to disable the parallel execution of tests per file in pants? We have some long running setup fixtures that we need for most tests but creating them for every file separately increased the overall test runtime drastically. Previously we have just used the

pytest-xdist

plugin with 8 workers, each worker then ran these fixtures once and the tests were evenly distributed across the workers. I am also open for other suggestions on how to potentially solve this issue.