https://pantsbuild.org/ logo
Join Slack
Powered by
I am trying to resolve all my Pants warnings and w...
# general
t
I am trying to resolve all my Pants warnings and would apprecaite some help on the following ones and how to resolve them. Appears on every pants command as far as I can see:
Copy code
13:08:58.07 [WARN] /my_repo/dist/export/python/virtualenvs/default/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
  warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
When I run a test, the following occurs:
Copy code
13:38:41.75 [WARN] /home/vscode/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/util/resources.py:28: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to <https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy> for migration advice.
  return resources.read_binary(package, resource)
When upgrading setuptools to 68.0.0 in my requirements file and setting pip_version = "23.0.1" in pants.toml, the first warning looks slightly different:
Copy code
13:47:39.45 [WARN] /home/vscode/.cache/nce/c55ee58a557d20bd4b109870e5a01b264c0d501ce817cce29502b2552903834d/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/init/options_initializer.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See <https://setuptools.pypa.io/en/latest/pkg_resources.html>
  import pkg_resources
@bitter-ability-32190 @happy-kitchen-89482 Is both related to the pants binary itself? I am wondering how this could be related to my setup at all.
b
The paths with the hash in the middle are pants itself. The first one, I couldn't tell. Looks like a user virtualenv.
h
Which Pants version?
t
2.16
h
Do you have any plugins? I'm not sure why changing setuptools in your requirements file would affect pants itself
t
No only a macro
Any ideas?
@bitter-ability-32190 The first is a symlink to the virtualenv as exported by 
pants export --resolve="['default', 'tools']"
Oh, I just noticed that both warnings disappear if I remove that symlink. Any idea why and how this can be avoided? Note that 
"/dist/"
is part of 
pants_ignore
h
Are you running pants with the virtualenv activated?
t
Not too sure to be honest. I am not explicitly calling the 
activate
script in the terminal. What I do is defining this symlinked path in my vscode settings file for PYTHONPATH, Pylance and tools (ruff, mypy, ...).
So what we do is activating the exported environment when a git command is executed from the VSCode SCM Panel: https://github.com/microsoft/vscode-python/issues/10165#issuecomment-1113857325. But the terminal I am using to run pants has no active virutalenv
Moving that symlink to another root folder, e.g. /usr/local/bin does not resolve it either.
Removing that workaround still shows me thows warnings 😞
@happy-kitchen-89482 @bitter-ability-32190 @enough-analyst-54434 Any ideas to further investigate?
b
Unless you can push a reproducible repo that I can poke at, I got nothing
e
@thousands-plumber-33255 Python is text, you can always take advantage of that. Edit the mystery line and add use of the traceback module. If stdout or stderr are hidden, output to a file. That's enough tool to hunt down most mystery sources of things in Python.
t
Full Traceback for the first warning:
Copy code
File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/bin/pants", line 8, in <module>
    sys.exit(main())
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 123, in main
    PantsLoader.main()
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 110, in main
    cls.run_default_entrypoint()
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 92, in run_default_entrypoint
    exit_code = runner.run(start_time)
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/bin/pants_runner.py", line 93, in run
    from pants.bin.local_pants_runner import LocalPantsRunner
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 31, in <module>
    from pants.init.options_initializer import OptionsInitializer
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/init/options_initializer.py", line 12, in <module>
    import pkg_resources
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/my_repo/dist/export/python/virtualenvs/default/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 132, in <module>
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
  File "/home/vscode/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/warnings.py", line 109, in _showwarnmsg
    sw(msg.message, msg.category, msg.filename, msg.lineno,
  File "/my_repo/dist/export/python/virtualenvs/default/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 127, in warn_with_traceback
    traceback.print_stack(file=log)
/my_repo/dist/export/python/virtualenvs/default/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py:132: DeprecationWarning: pkg_resources is deprecated as an API
  warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
Here is the other one:
Copy code
File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/dependency_inference/parse_python_dependencies.py", line 153, in general_parser_script
    script_digest = await get_scripts_digest(_scripts_package, ["general_dependency_visitor.py"])
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/dependency_inference/parse_python_dependencies.py", line 83, in get_scripts_digest
    scripts = [read_resource(scripts_package, filename) for filename in filenames]
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/dependency_inference/parse_python_dependencies.py", line 83, in <listcomp>
    scripts = [read_resource(scripts_package, filename) for filename in filenames]
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/util/resources.py", line 38, in read_resource
    return resources.read_binary(package, resource)
  File "/nefino_li/dist/export/python/virtualenvs/default/3.8/lib/python3.8/site-packages/importlib_resources/_legacy.py", line 18, in wrapper
    warnings.warn(
  File "/home/vscode/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/warnings.py", line 109, in _showwarnmsg
    sw(msg.message, msg.category, msg.filename, msg.lineno,
  File "/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/util/resources.py", line 34, in warn_with_traceback
    traceback.print_stack(file=log)
/home/vscode/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/util/resources.py:38: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to <https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy> for migration advice.
  return resources.read_binary(package, resource)
Okay here is the deal: I have not activated those "pants export"ed environments, but added its path to PYTHONPATH (For a few IDE reasons). It looks like pants is looking there then as well. How can this be avoided?
e
You'd submit a patch to fix Pants and eliminate its warned usages of 
pkg_resources
and 
importlib
.
Or ... continue down your PYTHON* env var route and export some warning filters.
t
Are you sure about the path? I thought the pants binary is self-contained, i.e. does not rely on a python installation. The problem seems to be that this pants venv pulls in outside dependencies that are present in the pythonpath
e
When a Python interpreter starts up, unless you pass it a flag like 
-E
, it will add PYTHONPATH entries, if present in the environment, to 
sys.path
. This is just how Python works. You'd have to be running code (like Pex) inside that Python interpreter that knew how to scrub such 
sys.path
entries to then get rid of them. The Pants installation runs in a normal venv without Pex intervening (at least through 2.17.x) and doing 
sys.path
scrubbing.
You can very easily just try this in any Python repl to prove it to yourself.
@thousands-plumber-33255 
PYTHONWARNINGS=...
Seems like a ~0 skin off your back solution if you'd rather not contribute Pants patches to fix this for everyone. Since you already are going through the effort to set 
PYTHONPATH=...
in your IDE, why not set one more env var?
3 Views
Open in Slack
PreviousNext
Powered by