Hi ! I am using pants build system for my projects...
# general
b
Hi ! I am using pants build system for my projects at zippin . I am using
attrs<24.0.0,>=23.2.0
package in my project. When I package the application into a
pex binary
and try to run the application. I get the following error.
Copy code
File "/Users/sidharthsingh/zippin_tech/dummy_project/main.py", line 4, in <module>
    from attrs import define
  File "/Users/sidharthsingh/projects/zippin_tech/dummy_project/.deps/attrs-23.2.0-py3-none-any.whl/attrs/__init__.py", line 3, in <module>
    from attr import (
ImportError: cannot import name 'AttrsInstance' from 'pex.vendor._vendored.attrs.attr' (/Users/sidharthsingh/zippin_tech/dummy_project/.bootstrap/pex/vendor/_vendored/attrs/attr/__init__.py)
Clearly something is wrong with the
sys.path
as the import should have happened from
/Users/sidharthsingh/projects/zippin_tech/dummy_project/.deps/attrs-23.2.0-py3-none-any.whl/attr/__init__.py
rather than
/Users/sidharthsingh/zippin_tech/dummy_project/.bootstrap/pex/vendor/_vendored/attrs/attr/__init__.py
Now, I after toying around for a while I changed argument
execution_mode
to
venv
in
pex_binary
target definition in
BUILD
file for this module. eg.
Copy code
# before: pex_binary(name="app", entry_point="main.py")
pex_binary(name="app", entry_point="main.py", execution_mode="venv")
This seemed to make it work, but I am not sure why. Is there an explanation to this behavior ?
b
Sorry for the trouble. Can you share your
pants.toml
? Of particular interest is the pants version and if you've customise the
[pex-cli]
section I'm looking for those because there's been some fixes to "sys.path leaks" in some relatively-recent PEX versions, e.g. https://github.com/pantsbuild/pex/blob/main/CHANGES.md#21140. It may be upgrading Pants and/or PEX is the fix you need
1
(I think the venv version works because PEX sets up a venv with just your code and then 'disappears' completely.)
b
Here you go !
Copy code
[GLOBAL]
pants_version = "2.18.0"
backend_packages = [
  ##################### SHELL #####################
  "pants.backend.shell",

  ##################### BUILD FILES #####################
  # Build file formatters
  "pants.backend.build_files.fmt.black",

  ##################### PROTOBUF #####################
  # "pants.backend.codegen.protobuf.lint.buf",
  "pants.backend.codegen.protobuf.python",

  ##################### PYTHON #####################
  # python-Linters; <https://www.pantsbuild.org/docs/python-linters-and-formatters>
  "pants.backend.python",
  "pants.backend.python.mixed_interpreter_constraints",
  # linters & type-checks
  # "pants.backend.python.lint.bandit",
  "pants.backend.experimental.python.lint.ruff",
  "pants.backend.python.lint.black",
  "pants.backend.python.lint.flake8",
  "pants.backend.python.lint.isort",
  # "pants.backend.python.lint.pylint",
  "pants.backend.python.lint.pyupgrade",
  "pants.backend.python.lint.docformatter",
  "pants.backend.python.lint.autoflake",
  "pants.backend.python.lint.pydocstyle",
  "pants.backend.python.typecheck.mypy",

  ##################### DOCKER #####################
  "pants.backend.docker",
  "pants.backend.docker.lint.hadolint",
]

pants_ignore.add = [
  "!*.mp4"
]

[source]
root_patterns = [
  "/src/*",
  "/build-support/*",
  "/3rdparty/*",
]

[python-protobuf]
mypy_plugin = true

[cli.alias]
diff = "--changed-since=origin/main --changed-dependents=direct"
diff-fmt = "--changed-since=origin/main --changed-dependents=direct update-build-files fmt fix"
diff-tests = "--test-use-coverage --changed-since=origin/main --changed-dependents=transitive test"
diff-checks = "--changed-since=origin/main --changed-dependents=direct update-build-files check lint fix fmt lint"
run-all-checks = "update-build-files check lint fix fmt ::"

[anonymous-telemetry]
enabled = false

[python]
interpreter_constraints = [">=3.10,<4"]
enable_resolves = true
default_resolve = "python-default"
default_run_goal_use_sandbox = true

[python.resolves]
python-default = "3rdparty/python/python-default.lock"
mypy = "3rdparty/python/mypy.lock"
flake8 = "3rdparty/python/flake8.lock"
pytest = "3rdparty/python/pytest.lock"
pylint = "3rdparty/python/pylint.lock"
isort = "3rdparty/python/isort.lock"
black = "3rdparty/python/black.lock"
ruff = "3rdparty/python/ruff.lock"


[python-repos]

path_mappings = ["WHEELS_DIR|%(buildroot)s/artifacts/prebuilt_wheels"]


[generate-lockfiles]
diff = true

[python-infer]
assets = true
unowned_dependency_behavior = "error"

[docformatter]
args = ["--wrap-summaries=100", "--wrap-descriptions=100"]

[flake8]
install_from_resolve = "flake8"
requirements = ["//3rdparty/python:flake8"]

[mypy]
install_from_resolve = "mypy"
requirements = ["//3rdparty/python:mypy"]

[isort]
install_from_resolve = "isort"
requirements = ["//3rdparty/python:isort"]

[black]
install_from_resolve = "black"
requirements = ["//3rdparty/python:black"]

[ruff]
install_from_resolve = "ruff"
requirements = ["//3rdparty/python:ruff"]

[pyupgrade]
args = ["--py310-plus"]

[test]
use_coverage = false

[pytest]
install_from_resolve = "pytest"
xdist_enabled = true
args = ["--no-header", "-vv", "-s"]
execution_slot_var = "TEST_EXECUTION_SLOT"
requirements = ["//3rdparty/python:pytest"]

[coverage-py]
interpreter_constraints = [">=3.10"]
global_report = true
report = ["xml", "console", "html"]


[docker]
# <https://www.pantsbuild.org/docs/docker#docker-authentication>
env_vars = [
  "DOCKER_DEFAULT_PLATFORM=linux/amd64",
  "DOCKER_CONFIG=%(homedir)s/.docker",
  "HOME=%(homedir)s",
]
build_args = ["BUILD_DATETIME", "GIT_COMMIT", "DOCKER_IMAGE_TAG"]
default_repository = "{parent_directory}-{directory}"
build_verbose = true

tools = ["docker-credential-gcloud", "dirname", "readlink", "python3"]

[docker.registries.gcr-zippindeveloper]
address = "<http://gcr.io/zippindeveloper|gcr.io/zippindeveloper>"
default = true
extra_image_tags = ["{build_args.DOCKER_IMAGE_TAG}"]
b
Ah, okay, so yeah, since there's no override of the
[pex-cli]
values, it's using the default, which is 2.1.137 in Pants 2.18.0, according to https://www.pantsbuild.org/docs/reference-pex-cli. You can upgrade PEX by either: • switching to pants 2.19.0rc4 (very close to the final release) • overriding it explicitly, something like:
Copy code
[pex-cli]
version = "v2.1.156"
known_versions = [
 ...
]
Where the
known_versions
should match the syntax of those docs, but with the new version, and the hash/file size of the new released artifact, from https://github.com/pantsbuild/pex/releases/tag/v2.1.156
1
b
Cool I will try this