refined-addition-53644
08/11/2022, 3:33 PMhundreds-father-404
08/11/2022, 3:34 PMrefined-addition-53644
08/11/2022, 3:34 PMhundreds-father-404
08/11/2022, 3:34 PMrefined-addition-53644
08/11/2022, 3:34 PMhundreds-father-404
08/11/2022, 3:36 PMrm -rf ~/.pants/
and rm -rf .pants.d src/rust/engine/target src/python/pants/engine/internal/native_engine.so
, which means rebuilding the engine 🐌refined-addition-53644
08/11/2022, 3:39 PMrm -rf ~/.cache/pants
?hundreds-father-404
08/11/2022, 3:39 PMsrc/python/pants/engine/internal/native_engine.so
first to see if that fixes itrefined-addition-53644
08/11/2022, 3:39 PMhundreds-father-404
08/11/2022, 3:55 PMenough-analyst-54434
08/11/2022, 4:17 PM/Users
) - before thinking too hard is this a damn M1?refined-addition-53644
08/11/2022, 4:17 PMenough-analyst-54434
08/11/2022, 4:17 PMrefined-addition-53644
08/11/2022, 4:17 PMenough-analyst-54434
08/11/2022, 4:17 PMrefined-addition-53644
08/11/2022, 4:18 PMenough-analyst-54434
08/11/2022, 4:18 PMfile src/python/pants/engine/internals/native_engine.so
say?refined-addition-53644
08/11/2022, 4:18 PMsrc/python/pants/engine/internals/native_engine.so: Mach-O 64-bit dynamically linked shared library x86_64
enough-analyst-54434
08/11/2022, 4:21 PMnm src/python/pants/engine/internals/native_engine.so | grep Py | grep -v _Z
- Mac may have a different tool for reading names - probably not nm
.refined-addition-53644
08/11/2022, 4:22 PMU _PyBaseObject_Type
U _PyBool_Type
U _PyBuffer_FromContiguous
U _PyBuffer_Release
U _PyBytes_AsString
U _PyBytes_FromStringAndSize
U _PyBytes_Size
U _PyCMethod_New
U _PyDict_GetItem
U _PyDict_Items
U _PyDict_New
U _PyDict_Next
U _PyDict_SetItem
U _PyDict_Size
U _PyErr_Fetch
U _PyErr_GivenExceptionMatches
U _PyErr_NewExceptionWithDoc
U _PyErr_NormalizeException
U _PyErr_Print
U _PyErr_PrintEx
U _PyErr_Restore
U _PyEval_EvalCode
U _PyEval_RestoreThread
U _PyEval_SaveThread
U _PyExc_AssertionError
U _PyExc_AttributeError
U _PyExc_BaseException
U _PyExc_BrokenPipeError
U _PyExc_BufferError
U _PyExc_Exception
U _PyExc_IOError
U _PyExc_KeyboardInterrupt
U _PyExc_OverflowError
U _PyExc_RuntimeError
U _PyExc_SystemError
U _PyExc_TypeError
U _PyExc_ValueError
U _PyException_GetCause
U _PyException_SetCause
U _PyFloat_AsDouble
U _PyGILState_Ensure
U _PyGILState_Release
U _PyImport_AddModule
U _PyImport_Import
00000000001bab30 T _PyInit_native_engine
U _PyInterpreterState_Main
U _PyIter_Next
U _PyList_Append
U _PyList_New
U _PyLong_AsLong
U _PyLong_AsUnsignedLongLong
U _PyLong_FromLong
U _PyLong_FromSsize_t
U _PyLong_FromUnsignedLongLong
U _PyModule_Create2
U _PyModule_GetDict
U _PyModule_GetName
U _PyNumber_Index
U _PyObject_Call
U _PyObject_DelItem
U _PyObject_Free
U _PyObject_GenericGetDict
U _PyObject_GenericSetDict
U _PyObject_GetAttr
U _PyObject_GetBuffer
U _PyObject_GetItem
U _PyObject_GetIter
U _PyObject_HasAttr
U _PyObject_IsInstance
U _PyObject_IsSubclass
U _PyObject_IsTrue
U _PyObject_Malloc
U _PyObject_Repr
U _PyObject_RichCompare
U _PyObject_SetAttr
U _PyObject_SetAttrString
U _PyObject_SetItem
U _PyObject_Size
U _PyObject_Str
U _PyObject_VectorcallMethod
U _PySequence_Check
U _PySequence_Size
U _PySet_Type
U _PyThreadState_New
U _PyTuple_New
U _PyTuple_SetItem
U _PyType_FromSpec
U _PyType_GenericAlloc
U _PyType_IsSubtype
U _PyType_Type
U _PyUnicode_AsEncodedString
U _PyUnicode_AsUTF8AndSize
U _PyUnicode_EncodeFSDefault
U _PyUnicode_FromStringAndSize
U _PyUnicode_InternInPlace
U _Py_CompileStringExFlags
U _Py_IsInitialized
U __PySet_NextEntry
U __Py_Dealloc
U __Py_FalseStruct
U __Py_NoneStruct
U __Py_NotImplementedStruct
U __Py_TrueStruct
enough-analyst-54434
08/11/2022, 4:25 PMU _PyCMethod_New
and your error is `E ImportError: dlopen(/private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-EgsQUA/src/python/pants/engine/internals/native_engine.so, 0x0002): symbol not found in flat namespace (_PyCMethod_New)`so the native_engine.so in the Pants sandbox != to the one in your code tree.hundreds-father-404
08/11/2022, 4:26 PMrefined-addition-53644
08/11/2022, 4:27 PM__run.sh
#!/bin/bash
# This command line should execute the same process as pants did internally.
export HOME=/Users/developer PATH=$'/Users/developer/.cache/pants/pants_dev_deps/Darwin.x86_64.i386.py39.venv/bin:/Users/developer/.sdkman/candidates/spark/current/bin:/Users/developer/.sdkman/candidates/scala/current/bin:/Users/developer/.sdkman/candidates/sbt/current/bin:/Users/developer/.sdkman/candidates/java/current/bin:/Users/developer/.nvm/versions/node/v16.13.0/bin:/usr/local/Cellar/pyenv-virtualenv/1.1.5/shims:/Users/developer/.pyenv/shims:/Users/developer/.pyenv/bin:/usr/local/opt/openssl/bin:/usr/local/opt/ruby/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Users/developer/.cargo/bin:/Users/developer/.fig/bin:/Users/developer/.local/bin:/Users/developer/Library/Application Support/Coursier/bin:/Applications/Visual Studio <http://Code.app/Contents/Resources/app/bin:/Users/developer/.local/bin:/Users/developer/Library/Python/3.9/bin:/Users/developer/Library/Application|Code.app/Contents/Resources/app/bin:/Users/developer/.local/bin:/Users/developer/Library/Python/3.9/bin:/Users/developer/Library/Application> Support/Coursier/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/Users/developer/.gem/ruby/3.0.0/bin:/Users/developer/shantanu/dx/bin:/Users/developer/shantanu/goprojects/bin:/usr/local/opt/go/libexec/bin:/Users/developer/.cargo/bin' PEX_EXTRA_SYS_PATH=$'.:src/python' PYENV_ROOT=/Users/developer/.pyenv PYTEST_ADDOPTS=$'--color=yes --junitxml=src.python.pants.backend.python.lint.flake8.rules_integration_test.py.tests.xml -o junit_family=xunit2' RUST_BACKTRACE=1
cd /private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-o1zjFN
./pytest_runner.pex_pex_shim.sh --no-header src/python/pants/backend/python/lint/flake8/rules_integration_test.py
ls -l /private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-sRoMTb/src/python/pants/engine/internals/native_engine.so
-rwxr-xr-x 1 developer staff 22847520 Aug 11 18:41 /private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-sRoMTb/src/python/pants/engine/internals/native_engine.so
enough-analyst-54434
08/11/2022, 4:43 PM./pytest_runner.pex_pex_shim.sh
maybe just cat its full contents here.refined-addition-53644
08/11/2022, 4:46 PMplatform darwin -- Python 3.7.9, pytest-7.0.1, pluggy-1.0.0
Although when running the script it also throws error
./pytest_runner.pex_pex_shim.sh
==================================================================================================================== test session starts =====================================================================================================================
platform darwin -- Python 3.7.9, pytest-7.0.1, pluggy-1.0.0
rootdir: /private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-sRoMTb, configfile: pyproject.toml
plugins: asyncio-0.18.3, metadata-2.0.1, html-3.1.1, icdiff-0.5, cov-3.0.0
asyncio: mode=auto
collected 0 items / 1 error
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Traceback (most recent call last):
File "/private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-sRoMTb/.cache/pex_root/venvs/3a41a0a3410ea86da89847adb93d8df66f4d4ff7/b2e7d54b77c9629156f1f0d75d836621e445af8c/pex", line 245, in <module>
sys.exit(func())
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/_pytest/config/__init__.py", line 188, in console_main
code = main()
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/_pytest/config/__init__.py", line 166, in main
config=config
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_callers.py", line 60, in _multicall
return outcome.get_result()
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_result.py", line 60, in get_result
raise ex[1].with_traceback(ex[2])
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/_pytest/main.py", line 315, in pytest_cmdline_main
return wrap_session(config, _main)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/_pytest/main.py", line 304, in wrap_session
session=session, exitstatus=session.exitstatus
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_callers.py", line 55, in _multicall
gen.send(outcome)
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/_pytest/terminal.py", line 792, in pytest_sessionfinish
outcome.get_result()
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_result.py", line 60, in get_result
raise ex[1].with_traceback(ex[2])
File "/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/lib/python3.7/site-packages/pluggy/_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File "/private/var/folders/9w/9_4n35r57d707zkrk86rvh9w0000gn/T/pants-sandbox-sRoMTb/src/python/pants/conftest.py", line 46, in pytest_sessionfinish
namespace_init_path.unlink()
File "/Users/developer/.pyenv/versions/3.7.9/lib/python3.7/pathlib.py", line 1309, in unlink
self._accessor.unlink(self)
FileNotFoundError: [Errno 2] No such file or directory: 'src/python/pants/__init__.py'
enough-analyst-54434
08/11/2022, 4:47 PMcat ./pytest_runner.pex_pex_shim.sh
and paste the contents here.refined-addition-53644
08/11/2022, 4:48 PM#!/bin/bash
set -euo pipefail
# N.B.: This relies on BASH_SOURCE which has been available since bash-3.0, released in
# 2004. It will either contain the absolute path of the venv script or it will contain
# the relative path from the CWD to the venv script. Either way, we know the venv script
# parent directory is the sandbox root directory.
SANDBOX_ROOT="${BASH_SOURCE%/*}"
function adjust_relative_paths() {
local value0="$1"
shift
if [ "${value0:0:1}" == "/" ]; then
# Don't relativize absolute paths.
echo "${value0}" "$@"
else
# N.B.: We convert all relative paths to paths relative to the sandbox root so
# this script works when run with a PWD set somewhere else than the sandbox
# root.
#
# There are two cases to consider. For the purposes of example, assume PWD is
# `/tmp/sandboxes/abc123/foo/bar`; i.e.: the rule API sets working_directory to
# `foo/bar`. Also assume `config/tool.yml` is the relative path in question.
#
# 1. If our BASH_SOURCE is `/tmp/sandboxes/abc123/pex_shim.sh`; so our
# SANDBOX_ROOT is `/tmp/sandboxes/abc123`, we calculate
# `/tmp/sandboxes/abc123/config/tool.yml`.
# 2. If our BASH_SOURCE is instead `../../pex_shim.sh`; so our SANDBOX_ROOT is
# `../..`, we calculate `../../config/tool.yml`.
echo "${SANDBOX_ROOT}/${value0}" "$@"
fi
}
export PATH='/Users/developer/.cache/pants/pants_dev_deps/Darwin.x86_64.i386.py39.venv/bin:/Users/developer/.sdkman/candidates/spark/current/bin:/Users/developer/.sdkman/candidates/scala/current/bin:/Users/developer/.sdkman/candidates/sbt/current/bin:/Users/developer/.sdkman/candidates/java/current/bin:/Users/developer/.nvm/versions/node/v16.13.0/bin:/usr/local/Cellar/pyenv-virtualenv/1.1.5/shims:/Users/developer/.pyenv/shims:/Users/developer/.pyenv/bin:/usr/local/opt/openssl/bin:/usr/local/opt/ruby/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Users/developer/.cargo/bin:/Users/developer/.fig/bin:/Users/developer/.local/bin:/Users/developer/Library/Application Support/Coursier/bin:/Applications/Visual Studio <http://Code.app/Contents/Resources/app/bin:/Users/developer/Library/Python/3.9/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/Users/developer/.gem/ruby/3.0.0/bin:/Users/developer/shantanu/dx/bin:/Users/developer/shantanu/goprojects/bin:/usr/local/opt/go/libexec/bin|Code.app/Contents/Resources/app/bin:/Users/developer/Library/Python/3.9/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/Users/developer/.gem/ruby/3.0.0/bin:/Users/developer/shantanu/dx/bin:/Users/developer/shantanu/goprojects/bin:/usr/local/opt/go/libexec/bin>' PEX_IGNORE_RCFILES=true PEX_ROOT=.cache/pex_root LC_CTYPE=UTF-8
export PEX_ROOT="$(adjust_relative_paths ${PEX_ROOT})"
execute_pex_args="$(adjust_relative_paths /Users/developer/.pyenv/versions/3.7.9/bin/python3.7) $(adjust_relative_paths pytest_runner.pex)"
target_venv_executable="$(adjust_relative_paths .cache/pex_root/venvs/3a41a0a3410ea86da89847adb93d8df66f4d4ff7/b2e7d54b77c9629156f1f0d75d836621e445af8c/pex)"
venv_dir="$(adjust_relative_paths .cache/pex_root/venvs/3a41a0a3410ea86da89847adb93d8df66f4d4ff7/b2e7d54b77c9629156f1f0d75d836621e445af8c)"
# Let PEX_TOOLS invocations pass through to the original PEX file since venvs don't come
# with tools support.
if [ -n "${PEX_TOOLS:-}" ]; then
exec ${execute_pex_args} "$@"
fi
# If the seeded venv has been removed from the PEX_ROOT, we re-seed from the original
# `--venv` mode PEX file.
if [ ! -e "${target_venv_executable}" ]; then
rm -rf "${venv_dir}" || true
PEX_INTERPRETER=1 ${execute_pex_args} -c ''
fi
exec "${target_venv_executable}" "$@"
enough-analyst-54434
08/11/2022, 4:49 PMhead -1 .cache/pex_root/venvs/3a41a0a3410ea86da89847adb93d8df66f4d4ff7/b2e7d54b77c9629156f1f0d75d836621e445af8c/pex
refined-addition-53644
08/11/2022, 4:49 PMenough-analyst-54434
08/11/2022, 4:49 PMrefined-addition-53644
08/11/2022, 4:49 PM#!/Users/developer/.cache/pants/named_caches/pex_root/venvs/s/a85f931a/venv/bin/python3.7 -sE
enough-analyst-54434
08/11/2022, 4:50 PM./pants
script to print out the `PY`it determines?refined-addition-53644
08/11/2022, 4:55 PMPY=/Users/developer/.pyenv/shims/python3.9
enough-analyst-54434
08/11/2022, 4:57 PMPY=python3.7 ./pants ...
to do your testing.hundreds-father-404
08/11/2022, 5:06 PMPY
thing works. I've been hardcoding PY
and [python].interpreter_constraints
the past 1+ year due to M1, so I have no idea if the defaults for the repo still are sensiblerefined-addition-53644
08/11/2022, 5:07 PMenough-analyst-54434
08/11/2022, 5:20 PMrefined-addition-53644
08/11/2022, 6:34 PM