pantsbuild-pants

in PyPI currently provides Python 3.7, 3.8, 3.9 wheels for the macosx/manylinux x86-64 platforms and 3.9 wheel for the macosx-arm64 platform. The missing 3.7 and 3.8 macosx-arm64 wheels are causing problems when I use

pants.backend.plugin_development

and

pants_requirements()

with

resolves_to_interpreter_constraints

option to split the lockfile for my own pants plugins under a mono-repo source tree: • Developers in macosx-arm64 platforms: they face missing package errors when Pants selects Python 3.7 or 3.8 as the interpreter of the "pants-plugins" resolve. They figure out that the fix is to change the target interpreter range of "pants-plugins" in

resolves_to_interpreter_constraints

setting to indicate 3.9 only. • After applying this fix, developers in linux-x86_64, macosx-x86_64 faces another error saying interpreter range mismatch with the requirement when running

./pants generate-lockfiles

. • As the result, I'm now telling developers in macosx-arm64 to remove all Python 3.7 and 3.8 versions from pyenv (as my

pants.toml

is configured to look for Python interpreters only from pyenv-installed ones). • This became problems when some developers want to use TensorFlow in macOS with Metal acceleration because the macOS TensorFlow package requires installation of Python 3.8. I'm getting tired of describing all these backgrounds everytime when people ask. Couldn't we just build and push the py37, py38 macosx-arm64 wheels of

pantsbuild-pants

to PyPI so that all target platforms have the same Python interpreter compatibility range? Also, it would make my life even easier if we have linux-aarch64 wheels as well, because for cases like above, I can just say that "use a arm64 Linux VM using UTM/QEMU/Parallels/VMWare/etc. for the best development exprerience with full speed" to Apple Silicon users.

hi all, i have a question about testing python wheels produced by pants. i have a monorepo / and in /ponder/BUILD I have a

python_distribution

rule producing a wheel we call “ponder”. the goal is for users to be able to

pip install

the wheel and then use the

ponder

library in other python code. I have a unit test in /pushdown_service/test/subdir_b/test_astype.py that depends on /pushdown_service/test/subdir_b/common.py, which has the line

import ponder.subdir

, but the

ponder

there comes directly from the source files, specifically via

ponder/subdir/__init__.py

. i want to do 2 things: 1. have the test use the ponder wheel instead of the source files within

ponder

. My goal here is to check that the wheel is being built correctly. I have already seen an error where we didn’t specify the wheel’s dependencies correctly, so the wheel was broken, but our tests, which use the ponder from the snowflake folder directly rather than from wheel, didn’t detect the failure. a. i tried adding

"ponder:ponder_wheel"

as a dependency to

python_tests

. but then I got the errors shown at the end of this message. what was wrong? the msising depdencies are all transitive dependencies of the ponder wheel 2. prevent the test from being able to access targets in

ponder/

that are not the wheel, such as the

ponder/subdir/_init_.py

target that it’s using currently is testing the wheel this way reasonable? are 1) and 2) possible?

pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/unzipped_pexes/0b173c486d327c83c634831eb9c4ac34b76e61ae.
Needed cp310-cp310-macosx_12_0_x86_64 compatible dependencies for:
 1: boto3
    Required by:
      ponder 0.0.3
    But this pex had no ProjectName(raw='boto3', normalized='boto3') distributions.
 2: fsspec[http]
    Required by:
      ponder 0.0.3
    But this pex had no ProjectName(raw='fsspec', normalized='fsspec') distributions.
 3: lz4==4.0.2
    Required by:
      ponder 0.0.3
    But this pex had no ProjectName(raw='lz4', normalized='lz4') distributions.
 4: modin@ git+<https://github.com/devin-petersohn/modin.git@2ad8813ccf3cb263857478df951b8c7b4b7d4396>
    Required by:
      ponder 0.0.3
    But this pex had no ProjectName(raw='modin', normalized='modin') distributions.
 5: rpyc==5.2.3
    Required by:
      ponder 0.0.3
    But this pex had no ProjectName(raw='rpyc', normalized='rpyc') distributions.

cc @clever-hamburger-59716

I keep getting this warning about PEX_ROOT inside a lambda docker image. This is after I have configured a custom location and provided read/write permissions. Not sure what I am missing

/function/service.pex/.bootstrap/pex/variables.py:582: PEXWarning: PEX_ROOT is configured as /pex_root but that path is un-writeable, falling back to a temporary PEX_ROOT of /tmp/tmpxcb25rwu which will hurt performance.

At the end it leads to PEX extracting packages multiple times and then server runs out of disk space.

If I remember correctly, I fixed this by setting the environment variable within the lambda:

PEX_ROOT=/tmp/.pex

yeah that worked. Thank you! Lost whole day after it 😞

"configured a custom location" - that's either using

--runtime-pex-root

on the Pex CLI or setting that env var somewhere, in the docker image using ENV instructions or in the lambda config as @billions-keyboard-33102 suggests. There are no other ways I'm aware of. How did you try to configure this @refined-addition-53644?

Yeah I was setting custom location inside docker

ENV PEX_ROOT=/pex_root

Hrm. Should work.

But even after giving all permissions pex was complaining about this location. Only thing worked was

/tmp/.pex

as suggested by Jake.

No clue what's up there.

yeah agree.

Do you know what user AWS runs the image with?

Probably not root.

I'm sure they have doc specifying this.

Yeah I couldn’t figure that out.

You could also figure it out with a few lines of code in your lambda.

Good thing to know for future. Voodoo works is always bad. Better to have solid works.

So from the docs preq section • The container image must be able to run on a read-only file system. Your function code can access a writable

/tmp

directory with between 512 MB and 10,240 MB, in 1-MB increments, of storage. • The default Lambda user must be able to read all the files required to run your function code. Lambda follows security best practices by defining a default Linux user with least-privileged permissions. Verify that your application code does not rely on files that other Linux users are restricted from running. I don’t know how to fetch this default Lambda user. https://docs.aws.amazon.com/lambda/latest/dg/images-create.html

Well bullet point 1 is enough.

Says it right there, use /tmp

Soo, @refined-addition-53644 and @billions-keyboard-33102 if you're using a container you should definitely do something like this instead: https://pex.readthedocs.io/en/v2.1.117/recipes.html#pex-app-in-a-container That pre-installs the PEX and pre-compiles it's .pyc. Lower latency cold start and no need for writable PEX_ROOT or any Pex anything at runtime.

I need some help getting my first-party type checks working. I have a directory structured like this:

src
├── common
│   ├── auth
|       ├── BUILD
|       ├── auth1.py
|       ├── auth2.py
│       └── mypy-stubs
|           ├── BUILD
|           ├── auth1.pyi
│           └── auth2.pyi
├── services
│   ├── app
|       ├── BUILD
│       └── app.py

I want to run type-checking on

app

, which imports `auth1`/`auth2`. When I list dependencies for

auth

, I see the `pyi`s, but I don't see them in dependencies for

app

. I have tried explicitly listing them app's

BUILD

, and they will be listed as dependencies when I do, but the type check still fails with

error: Cannot find implementation or library stub for module named "common.auth.auth1"

(same error for

auth2

). I feel like I've gotten this to work in the past - what am I missing here?

@enough-analyst-54434 So I had some issues compiling the dependencies. https://pantsbuild.slack.com/archives/C046T6T9U/p1668701343071699?thread_ts=1668701343.071699&cid=C046T6T9U

Question about debugging in VSCode. I set up a Python: Remote Attach VSCode configuration and launched my app with

pants run --debug-adapter apps/python/example_app/app.py

which launches and waits with

Launching debug adapter at '127.0.0.1:5678...'

I then started the remote attach config. The program continues and prints to the VSCode Debug Console but does not stop on break points. So I tried PyCharm and was able to successfully debug by hosting a debug server and including in my python code:

import pydevd_pycharm
pydevd_pycharm.settrace('localhost', port=5150, stdoutToServer=True, stderrToServer=True)

It does not map to the original source file but I'm able to step through with the debugger in PyCharm. I then tried to get VSCode working and found that with the above

pydevd_pycharm

code in place, I can debug in VSCode as well. It also does not map to the original source file but I can connect to the remote session and step through the code. If I remove or comment the

pydevd_pycharm

code, I see the original behavior where the console messages print to the debug console but execution is not halted. Thoughts on what I may be missing?

Hey Pants Team, curious if anyone has seen this before in CI:

2022-11-30T22:41:52Z] 22:41:52.07 [ERROR] Permission denied (publickey).
[2022-11-30T22:41:52Z] fatal: Could not read from remote repository.
[2022-11-30T22:41:52Z]
[2022-11-30T22:41:52Z] Please make sure you have the correct access rights
[2022-11-30T22:41:52Z] and the repository exists.
[2022-11-30T22:41:52Z] Permission denied (publickey).
[2022-11-30T22:41:52Z] fatal: Could not read from remote repository.
[2022-11-30T22:41:52Z]
[2022-11-30T22:41:52Z] Please make sure you have the correct access rights
[2022-11-30T22:41:52Z] and the repository exists.
[2022-11-30T22:41:52Z] fatal: unable to read 6a8276e5ec50a3dfc850f4c636e9bae80199aa73
[2022-11-30T22:41:52Z] Traceback (most recent call last):
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/bin/daemon_pants_runner.py", line 131, in single_daemonized_run
[2022-11-30T22:41:52Z]     runner = LocalPantsRunner.create(
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 158, in create
[2022-11-30T22:41:52Z]     specs = calculate_specs(
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/init/specs_calculator.py", line 68, in calculate_specs
[2022-11-30T22:41:52Z]     changed_files = tuple(changed_options.changed_files(maybe_git_worktree.git_worktree))
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/vcs/changed.py", line 127, in changed_files
[2022-11-30T22:41:52Z]     git_worktree.changed_files(
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/vcs/git.py", line 91, in changed_files
[2022-11-30T22:41:52Z]     committed_changes = self._git_binary._invoke_unsandboxed(
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/core/util_rules/system_binaries.py", line 384, in _invoke_unsandboxed
[2022-11-30T22:41:52Z]     self._check_result(cmd, process.returncode, err.decode())
[2022-11-30T22:41:52Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/core/util_rules/system_binaries.py", line 393, in _check_result
[2022-11-30T22:41:52Z]     raise GitBinaryException(failure_msg or f"{cmd_str} failed with exit code {result}")
[2022-11-30T22:41:52Z] pants.core.util_rules.system_binaries.GitBinaryException: Permission denied (publickey).
[2022-11-30T22:41:52Z] fatal: Could not read from remote repository.
[2022-11-30T22:41:52Z]
[2022-11-30T22:41:52Z] Please make sure you have the correct access rights
[2022-11-30T22:41:52Z] and the repository exists.
[2022-11-30T22:41:52Z] Permission denied (publickey).
[2022-11-30T22:41:52Z] fatal: Could not read from remote repository.
[2022-11-30T22:41:52Z]
[2022-11-30T22:41:52Z] Please make sure you have the correct access rights
[2022-11-30T22:41:52Z] and the repository exists.
[2022-11-30T22:41:52Z] fatal: unable to read 6a8276e5ec50a3dfc850f4c636e9bae80199aa73
[2022-11-30T22:41:52Z]
[2022-11-30T22:41:52Z]
[2022-11-30T22:41:52Z] Use -ldebug for more logs.
[2022-11-30T22:41:52Z] See <https://www.pantsbuild.org/v2.14/docs/troubleshooting> for common issues.
[2022-11-30T22:41:52Z] Consider reaching out for help: <https://www.pantsbuild.org/v2.14/docs/getting-help>

the

--change-since

option is failing randomly in builds.

Hello 👋 I am having a problem running a unit test in

pyspark

that uses a user-defined function (UDF). I’ve created a small toy repo to replicate this problem: https://github.com/skyxie/pants-test The

README.md

has the reproduction steps, but to break it down 1. In

conftest.py

there is a

spark

fixture to create a local spark session. 2. In

src/my_module/__init__.py

there are 2 UDFs, one that does not include a 3rd-party dependency and one that does. 3rd-party dependencies are specified in

3rdparty/requirements.txt

3. In

test/test_my_module.py

I have unit tests for each of the 2 UDFs. If run these unit tests using

PYTHONPATH=src pytest test/test_my_module.py

then both tests pass, but if I run them using

./pants test ::

, then the UDF that uses the 3rd-party dependency fails. The error I get on my local MacOSX is cryptic

22/11/30 23:14:36 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1) (ip-192-168-0-26.ec2.internal executor driver): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed)

But if I run

./pants test ::

in a docker container with

python

and

openjdk8

installed, then I get a more interesting error:

22/12/01 04:32:15 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1) (6f12a8103121 executor driver): org.apache.spark.api.python.PythonException: Traceback (most recent call last):
  ... # Truncated for brevity
  File "/root/.cache/pants/named_caches/pex_root/venvs/ccde778f33dcd85107fc1c69e63d0aece19f2091/1f95d470b7a8fe245d4aa248c6b523b37b0dc73f/lib/python3.8/site-packages/pyspark/python/lib/pyspark.zip/pyspark/serializers.py", line 430, in loads
    return pickle.loads(obj, encoding=encoding)
  File "/root/.cache/pants/named_caches/pex_root/venvs/ccde778f33dcd85107fc1c69e63d0aece19f2091/1f95d470b7a8fe245d4aa248c6b523b37b0dc73f/lib/python3.8/site-packages/pyspark/python/lib/pyspark.zip/pyspark/cloudpickle/cloudpickle.py", line 562, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'tldextract'

which suggests that the spark worker attempts to load a pickle, but cannot import a module referenced in the pickle. My hypothesis for why this happens is that spark session spawns a worker that is in a different python virtualenvironment from the one running in pants and therefore cannot load the dependencies.

I am a bit puzzled at pants' documentation for interpreter compatibility in python. I mean: the doc doesn't say explicitly what the constraints are used for. I suppose it is for

pants

to check that the running interpreter is included in the constraint specified by

interpreter_constraints

, correct? What happens when 1/ the top-level constraint is overridden by more specific constraint in a subset of the repo and 2/ the running interpreter is the most general one (i.e. it doesn't honor the more specific constraint)? Does

pants

automatically skips the targets whose constraint is not satisfied by the interpreter? Or does

pants

build the interpreter on its own when seeing the constraints? I can't remember how

pants

bootstrapped the first times I used it 😅

Hi, how would you install some packages inside the

docker_environment

specified image? For example I need to install gcc inside base python image used by this env. https://www.pantsbuild.org/v2.15/docs/environments#use-a-docker_environment-to-build-the-inputs-to-a-docker_image

hi what doe the

--check

commands do. following the CI tutorial and got the following output when running

./pants tailor --check update-build-files --check

[WARN] No arguments specified with `./pants tailor`, so the goal will do nothing.

Instead, you should provide arguments like this:

  * `./pants tailor ::` to run on everything
  * `./pants tailor dir::` to run on `dir` and subdirs
  * `./pants tailor dir` to run on `dir`
  * `./pants --changed-since=HEAD tailor` to only run on changed and new files
13:28:03.85 [WARN] No arguments specified with `./pants update-build-files`, so the goal will do nothing.

Instead, you should provide arguments like this:

  * `./pants update-build-files ::` to run on everything
  * `./pants update-build-files dir::` to run on `dir` and subdirs
  * `./pants update-build-files dir` to run on `dir`
  * `./pants update-build-files dir/BUILD` to run on that single BUILD file
  * `./pants --changed-since=HEAD update-build-files` to run only on changed BUILD files