Hey Pants Team We are upgrading from 2 13 gt 2 14 and runnin Pants #general

Hey Pants Team, We are upgrading from 2.13 -> 2...

rapid-bird-79300

11/15/2022, 10:45 PM

Hey Pants Team, We are upgrading from 2.13 -> 2.14 and running into this issue:

Copy code

[2022-11-15T22:41:38Z] ModuleNotFoundError: No module named '_sqlite3'

this is when running lint in CI. Will paste some logs in the thread.

✅ 1

rapid-bird-79300

11/15/2022, 10:45 PM

Copy code

[2022-11-15T22:41:38Z] 22:41:38.97 [INFO] Completed: Building black.pex from black_default.lock
[2022-11-15T22:41:38Z] 22:41:38.98 [ERROR] 1 Exception encountered:
[2022-11-15T22:41:38Z]
[2022-11-15T22:41:38Z] Engine traceback:
[2022-11-15T22:41:38Z]   in select
[2022-11-15T22:41:38Z]   in pants.core.goals.fmt.fmt
[2022-11-15T22:41:38Z]   in pants.core.goals.fmt.fmt_target_batch
[2022-11-15T22:41:38Z]   in pants.backend.python.lint.black.rules.black_fmt (black)
[2022-11-15T22:41:38Z]   in pants.backend.python.util_rules.pex.create_venv_pex (black.pex)
[2022-11-15T22:41:38Z]   in pants.backend.python.util_rules.pex.build_pex (black.pex)
[2022-11-15T22:41:38Z]   in pants.engine.process.fallible_to_exec_result_or_raise
[2022-11-15T22:41:38Z] Traceback (most recent call last):
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.0_py39/lib/python3.9/site-packages/pants/engine/process.py", line 275, in fallible_to_exec_result_or_raise
[2022-11-15T22:41:38Z]     raise ProcessExecutionFailure(
[2022-11-15T22:41:38Z] pants.engine.process.ProcessExecutionFailure: Process 'Building black.pex from black_default.lock' failed with exit code 1.
[2022-11-15T22:41:38Z] stdout:
[2022-11-15T22:41:38Z]
[2022-11-15T22:41:38Z] stderr:
[2022-11-15T22:41:38Z] Traceback (most recent call last):
[2022-11-15T22:41:38Z]   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
[2022-11-15T22:41:38Z]     return _run_code(code, main_globals, None,
[2022-11-15T22:41:38Z]   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
[2022-11-15T22:41:38Z]     exec(code, run_globals)
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/__main__.py", line 103, in <module>
[2022-11-15T22:41:38Z]     bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex_bootstrapper.py", line 601, in bootstrap_pex
[2022-11-15T22:41:38Z]     pex.PEX(entry_point).execute()
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex.py", line 541, in execute
[2022-11-15T22:41:38Z]     sys.exit(self._wrap_coverage(self._wrap_profiling, self._execute))
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex.py", line 448, in _wrap_coverage
[2022-11-15T22:41:38Z]     return runner(*args)
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex.py", line 479, in _wrap_profiling
[2022-11-15T22:41:38Z]     return runner(*args)
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex.py", line 568, in _execute
[2022-11-15T22:41:38Z]     return self.execute_entry(
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex.py", line 750, in execute_entry
[2022-11-15T22:41:38Z]     return self.execute_entry_point(entry_point)
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/pex.py", line 767, in execute_entry_point
[2022-11-15T22:41:38Z]     runner = entry_point.resolve()
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/unzipped_pexes/9ffa289649151e7d6da966ba4cdd7c2aae242f1e/.bootstrap/pex/dist_metadata.py", line 726, in resolve
[2022-11-15T22:41:38Z]     module = importlib.import_module(self.module)
[2022-11-15T22:41:38Z]   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
[2022-11-15T22:41:38Z]     return _bootstrap._gcd_import(name[level:], package, level)
[2022-11-15T22:41:38Z]   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
[2022-11-15T22:41:38Z]   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
[2022-11-15T22:41:38Z]   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
[2022-11-15T22:41:38Z]   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
[2022-11-15T22:41:38Z]   File "<frozen importlib._bootstrap_external>", line 855, in exec_module
[2022-11-15T22:41:38Z]   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/installed_wheels/735ba74cef654ed9306ec06e9e2d860eca7ef7df89bb666e69046a64904b4300/pex-2.1.108-py2.py3-none-any.whl/pex/bin/pex.py", line 36, in <module>
[2022-11-15T22:41:38Z]     from pex.resolve.configured_resolver import ConfiguredResolver
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/installed_wheels/735ba74cef654ed9306ec06e9e2d860eca7ef7df89bb666e69046a64904b4300/pex-2.1.108-py2.py3-none-any.whl/pex/resolve/configured_resolver.py", line 6, in <module>
[2022-11-15T22:41:38Z]     from pex import resolver
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/installed_wheels/735ba74cef654ed9306ec06e9e2d860eca7ef7df89bb666e69046a64904b4300/pex-2.1.108-py2.py3-none-any.whl/pex/resolver.py", line 30, in <module>
[2022-11-15T22:41:38Z]     from pex.resolve.downloads import get_downloads_dir
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/installed_wheels/735ba74cef654ed9306ec06e9e2d860eca7ef7df89bb666e69046a64904b4300/pex-2.1.108-py2.py3-none-any.whl/pex/resolve/downloads.py", line 15, in <module>
[2022-11-15T22:41:38Z]     from pex.resolve import locker
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/installed_wheels/735ba74cef654ed9306ec06e9e2d860eca7ef7df89bb666e69046a64904b4300/pex-2.1.108-py2.py3-none-any.whl/pex/resolve/locker.py", line 27, in <module>
[2022-11-15T22:41:38Z]     from pex.resolve.pep_691.fingerprint_service import FingerprintService
[2022-11-15T22:41:38Z]   File "/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/installed_wheels/735ba74cef654ed9306ec06e9e2d860eca7ef7df89bb666e69046a64904b4300/pex-2.1.108-py2.py3-none-any.whl/pex/resolve/pep_691/fingerprint_service.py", line 7, in <module>
[2022-11-15T22:41:38Z]     import sqlite3
[2022-11-15T22:41:38Z]   File "/usr/local/lib/python3.9/sqlite3/__init__.py", line 23, in <module>
[2022-11-15T22:41:38Z]     from sqlite3.dbapi2 import *
[2022-11-15T22:41:38Z]   File "/usr/local/lib/python3.9/sqlite3/dbapi2.py", line 27, in <module>
[2022-11-15T22:41:38Z]     from _sqlite3 import *
[2022-11-15T22:41:38Z] ModuleNotFoundError: No module named '_sqlite3'
[2022-11-15T22:41:38Z]
[2022-11-15T22:41:38Z]
[2022-11-15T22:41:38Z]
[2022-11-15T22:41:38Z] Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

witty-crayon-22786

11/15/2022, 11:15 PM

how is the python interpreter that you are using built?

rapid-bird-79300

11/15/2022, 11:21 PM

yeah it's built from source in an packer build job

rapid-bird-79300

11/15/2022, 11:21 PM

I added

sqlite-develop

but notice this change between the two versiosn

witty-crayon-22786

11/15/2022, 11:22 PM

likely due to https://github.com/pantsbuild/pex/pull/1882

👍 1

witty-crayon-22786

11/15/2022, 11:23 PM

whether requiring

sqlite3

in a python interpreter is a reasonable thing i’m actually not sure… if it doesn’t feel reasonable, filing a PEX ticket about that would be helpful

happy-kitchen-89482

11/15/2022, 11:55 PM

sqlite3 is part of the standard library, even in Python 2.7

happy-kitchen-89482

11/15/2022, 11:56 PM

So it seems reasonable to expect it to exist?

👍 2

rapid-bird-79300

11/16/2022, 5:06 PM

added sqlite fixed the issue. I agree it seems reasonable to expect this to be present. Thanks for the help here!

6 Views

Open in Slack

Previous Next