freezing-fall-2672
09/14/2023, 1:18 PMfreezing-fall-2672
09/14/2023, 1:18 PMINFO:add_license_files:adding license file 'LICENCE'
INFO:execute:writing manifest file 'src/M2Crypto.egg-info/SOURCES.txt'
INFO:run_command:running build_ext
INFO:build_extension:building 'M2Crypto._m2crypto' extension
INFO:swig_sources:swigging src/SWIG/_m2crypto.i to src/SWIG/_m2crypto_wrap.c
INFO:spawn:swig -python -py3 -I/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.3/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include "-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)" -I/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.3/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include "-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)" -I/Users/elmcrest/.cache/pants/named_caches/pex_root/venvs/839dd85d778df5927a872ace6e92e15b4f301d3d/db8266b13362d394ecbbe9b1d67bbc2c832a080c/include -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -I/usr/include/openssl -includeall -modern -builtin -outdir /Users/elmcrest/.cache/pants/named_caches/pex_root/pip_cache/.tmp/pip-req-build-_xs4lqvc/src/M2Crypto -o src/SWIG/_m2crypto_wrap.c src/SWIG/_m2crypto.i
Deprecated command line option: -py3. Ignored, this option is no longer supported.
Deprecated command line option: -modern. Ignored, this option is now always on.
src/SWIG/_m2crypto.i:62: Error: Unable to find 'openssl/opensslv.h'
src/SWIG/_m2crypto.i:68: Error: Unable to find 'openssl/safestack.h'
src/SWIG/_evp.i:12: Error: Unable to find 'openssl/opensslconf.h'
src/SWIG/_rc4.i:5: Error: Unable to find 'openssl/opensslconf.h'
src/SWIG/_ec.i:7: Error: Unable to find 'openssl/opensslconf.h'
error: command '/opt/homebrew/bin/swig' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for M2Crypto
ERROR: Failed to build one or more wheels
freezing-fall-2672
09/14/2023, 1:24 PMpants --python-native-code-cpp-flags="-I$(brew --prefix openssl)/include" --python-native-code-ld-flags="-L$(brew --prefix openssl)/lib" test foo/bar/asset_management/tests/tests_asset.py
doesn't work for me, toofreezing-fall-2672
09/14/2023, 1:24 PMfreezing-fall-2672
09/14/2023, 1:29 PMpants test --python-native-code-cpp-flags="['-I/opt/homebrew/opt/openssl@3/include']" --python-native-code-ld-flags="['-L/opt/homebrew/opt/openssl@3/lib']" foo/bar/asset_management/tests/tests_asset.py
or
pants test --python-native-code-cpp-flags="['-I /opt/homebrew/opt/openssl@3/include']" --python-native-code-ld-flags="['-L /opt/homebrew/opt/openssl@3/lib']" foo/bar/asset_management/tests/tests_asset.py
I wasn't sure if -L /
or -L/
is correct, both don't work anywaysfreezing-fall-2672
09/14/2023, 1:36 PMpip install M2Crypto --no-cache-dir
freezing-fall-2672
09/14/2023, 1:39 PMCFLAGS="-I$(brew --prefix openssl)/include" LDFLAGS="-L$(brew --prefix openssl)/lib" SWIG_FEATURES="-I$(brew --prefix openssl)/include" pip install M2Crypto --no-cache-dir
enough-analyst-54434
09/14/2023, 1:41 PMfreezing-fall-2672
09/14/2023, 1:43 PMpants test --extra-env-vars="['CFLAGS="-I$(brew --prefix openssl)/include"','LDFLAGS="-L$(brew --prefix openssl)/lib"','SWIG_FEATURES="-I$(brew --prefix openssl)/include"']" foo/bar/asset_management/tests/tests_asset.py
so far no luck though 😕freezing-fall-2672
09/14/2023, 1:44 PMfreezing-fall-2672
09/14/2023, 1:49 PMpants test --python-native-code-cpp-flags="['-I/opt/homebrew/opt/openssl@3/include']" --python-native-code-ld-flags="['-L/opt/homebrew/opt/openssl@3/lib']" --extra-env-vars="['SWIG_FEATURES=-I/opt/homebrew/opt/openssl@3/include']" foo/bar/asset_management/tests/tests_asset.py --keep-sandboxes=always
I get the cpp and ld flags into the sandbox - at least I can see something in the __run.sh
file but the SWIG_FEATURES is missingfreezing-fall-2672
09/14/2023, 1:56 PMfreezing-fall-2672
09/14/2023, 1:56 PMenough-analyst-54434
09/14/2023, 2:03 PMfreezing-fall-2672
09/14/2023, 2:03 PMfreezing-fall-2672
09/14/2023, 2:04 PMenough-analyst-54434
09/14/2023, 2:04 PMfreezing-fall-2672
09/14/2023, 2:05 PMcurved-television-6568
09/14/2023, 2:42 PMcurved-television-6568
09/14/2023, 2:44 PM[subprocess-environment]
env_vars.add = [
"GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1",
"GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1",
]
[python-native-code]
cpp_flags = "-I/opt/homebrew/opt/openssl@3/include"
ld_flags = "-L/opt/homebrew/opt/openssl@3/lib"
freezing-fall-2672
09/14/2023, 2:47 PMfreezing-fall-2672
09/14/2023, 2:47 PMfreezing-fall-2672
09/14/2023, 3:06 PMfreezing-fall-2672
09/15/2023, 5:26 AM[subprocess-environment]
section in the pant.toml config file for pants gives the opportunity to configure things for the subprocess env (as the name suggests) it uses to execute target language specific functionality, f.e. python tests or linters - and I guess the same if you'd use JS or some other supported language.
the env_vars.add
variable is a list of env vars which will be added to the subprocess env. here I had to use it to provided the needed SWIG_FEATURES
flag.
the [python-native-code]
section in the pants.toml config file for pants gives the opportunity to configure the cpp_flag and ld_flag if python pip needs to build a package from source. The -L
flag points the C/C++ compiler to 'to shared libraries, directory containing *.so, *.dll or *.dlyb depending on the current operating system.' while the -I
flag points the compiler to header files. Both is implemented in a PATH
fashion, so the compiler knows where to look for things.
Why this solved my issue here:
pants was building the sandbox with my 3rd party requirements and while doing so it hit M2Crypto
which doesn't provide a wheel (precompiled binary) which could just be downloaded instead of compiling locally from source - so it had no other chance than compiling it. since pants is executing things in a subprocess with it's own shell and inline env vars aren't passed along so there needs to be different options to do so.
I hope that's fulfilling your requirements @enough-analyst-54434 - let me know if something is harshly wrong and of course if parts or not correct enough so to speak.
Besides that I would expect that the --test-extra-env-vars
or effectively the same --extra-env-vars
would still work here for all three environment variables.
Considering that pip seems to create an isolated build environment since version 10 (with PEP 517, right?) where env vars from the main pip process are passed along, I would assume that this, for me, unexpected behavior is related to how pants internals are working and specifically the rust side of things - which is also the side I have the least knowledge about so that's an easy blame for me 🙂
happy friday and until my next question, I hope 😆curved-television-6568
09/15/2023, 1:46 PMpytest
) any requirements needed to run your tests have already been installed, so it’s too late to affect any third party building using that option.freezing-fall-2672
09/15/2023, 1:47 PMcurved-television-6568
09/15/2023, 1:48 PMsubprocess-environment
sounds very generic, but it really only applies for a subset of invoked subprocesses (it’s a case by case, as they need to respect this option per invocation site) which is slightly unfortunate situation, but good to be aware of I think.. 😉curved-television-6568
09/15/2023, 1:48 PMfreezing-fall-2672
09/15/2023, 1:49 PMsubprocess-environment
isn't implemented for everything - that's what you mean with case by case, right?curved-television-6568
09/15/2023, 1:50 PMfreezing-fall-2672
09/15/2023, 1:50 PM