Hi everyone! I am using pants v2.16.0 for python versions >=3.6.2, <4 for a project and I'm encountering the following error while running

./pants export ::

.

Exception:
Traceback (most recent call last):
  File "dist/export/python/virtualenvs/python-default/3.6.15/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "dist/export/python/virtualenvs/python-default/3.6.15/lib/python3.6/site-packages/pip/_internal/commands/uninstall.py", line 48, in run
    with self._build_session(options) as session:
  File "dist/export/python/virtualenvs/python-default/3.6.15/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 76, in _build_session
    if options.cache_dir else None
  File "/usr/local/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not int
Failed to uninstall venv versions of setuptools: Command '['dist/export/python/virtualenvs/python-default/3.6.15/bin/python3.6', '-m', 'pip', 'uninstall', '-y', 'setuptools']' returned non-zero exit status 2.
12:17:23.82 [ERROR] 1 Exception encountered:

Engine traceback:
  in `export` goal

ExportError: Failed to write mutable virtualenv for python-default (using Python 3.6.15) to dist/export/python/virtualenvs/python-default/3.6.15

It seems to be an error associated with setuptools installation for python3.6 here is the pants.toml file used

[GLOBAL]
pants_version = "2.16.0"
backend_packages.add = [
  "pants.backend.python",
  "pants.backend.python.lint.docformatter",
  "pants.backend.python.lint.black",
  "pants.backend.python.lint.flake8",
  "pants.backend.python.lint.isort",
  "pants.backend.python.typecheck.mypy",
]

[source]
root_patterns = ["/edge/python"]

[python]
# We set the interpreter constraints to some version of python 3.6 to enable compatibility with edge devices.
# Individual targets can override this with the `interpreter_constraints` field. See
# <https://www.pantsbuild.org/docs/python-interpreter-compatibility>.
interpreter_constraints = ["CPython>=3.6.2,<4"]

# Enable the "resolves" mechanism, which turns on lockfiles for user code. See
# <https://www.pantsbuild.org/docs/python-third-party-dependencies>. This also adds the
# `generate-lockfiles` goal for Pants to generate the lockfile for you, although in some
# situations you may need to manually generate it, as explained in the docs.
enable_resolves = true

[python.resolves]
python-default = "edge/python/3rdparty/lockfile.txt"
isort = "edge/python/3rdparty/isort.lock"
docformatter = "edge/python/3rdparty/docformatter.lock"

[python-bootstrap]
# We search for interpreters both on the $PATH and in the `$(pyenv root)/versions` folder.
#  If you're using macOS, you may want to leave off the <PATH> entry to avoid using the
#  problematic system Pythons. See
#  <https://www.pantsbuild.org/docs/python-interpreter-compatibility#changing-the-interpreter-search-path>.
search_path = ["<PATH>", "<PYENV>"]

[flake8]
lockfile = "edge/python/3rdparty/flake8/lockfile.txt"

[pytest]
lockfile = "edge/python/3rdparty/pytest/lockfile.txt"
extra_requirements.add = ["pytest-asyncio==0.16.0"]

[isort]
install_from_resolve = "isort"

[docformatter]
install_from_resolve = "docformatter"

I don't repro a facsimile of the setup (Pants 2.16.0 uses Pex 2.1.134 like so to generate a mutable export venv):

$ pex --python python3.6 pex==2.1.134 -cpex -- cowsay==5.0 -ccowsay --include-tools -o cowsay.pex
$ PEX_TOOLS=1 python3.6 ./cowsay.pex venv --pip --collisions-ok cowsay-3.6.venv
$ ls -l cowsay-3.6.venv/lib/python3.6/site-packages/
total 44
-rw-r--r-- 1 jsirois jsirois  230 Oct  5 13:25 PEX_EXTRA_SYS_PATH.pth
drwxr-xr-x 2 jsirois jsirois 4096 Oct  5 13:25 __pex__
drwxr-xr-x 2 jsirois jsirois 4096 Oct  5 13:25 __pycache__
drwxr-xr-x 3 jsirois jsirois 4096 Oct  5 13:25 cowsay
drwxr-xr-x 2 jsirois jsirois 4096 Oct  5 13:25 cowsay-5.0.dist-info
-rw-r--r-- 1 jsirois jsirois  126 Oct  5 13:25 easy_install.py
drwxr-xr-x 5 jsirois jsirois 4096 Oct  5 13:25 pip
drwxr-xr-x 2 jsirois jsirois 4096 Oct  5 13:25 pip-18.1.dist-info
drwxr-xr-x 5 jsirois jsirois 4096 Oct  5 13:25 pkg_resources
drwxr-xr-x 6 jsirois jsirois 4096 Oct  5 13:25 setuptools
drwxr-xr-x 2 jsirois jsirois 4096 Oct  5 13:25 setuptools-40.6.2.dist-info
$ cowsay-3.6.venv/bin/cowsay 'Moo!'
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

@lemon-kangaroo-17191 Can you run through those same steps and provide the output like I did?

@enough-analyst-54434

$ pex --python python3.6 pex==2.1.134 -cpex -- cowsay==5.0 -ccowsay --include-tools -o cowsay.pex
$ PEX_TOOLS=1 python3.6 ./cowsay.pex venv --pip --collisions-ok cowsay-3.6.venv
$ ls -l cowsay-3.6.venv/lib/python3.6/site-packages/
total 44
drwxrwxr-x 2 raghak raghak 4096 Oct  5 13:49 cowsay
drwxrwxr-x 2 raghak raghak 4096 Oct  5 13:49 cowsay-5.0.dist-info
-rw-rw-r-- 1 raghak raghak  126 Oct  5 13:49 easy_install.py
drwxrwxr-x 2 raghak raghak 4096 Oct  5 13:49 __pex__
-rw-rw-r-- 1 raghak raghak  230 Oct  5 13:49 PEX_EXTRA_SYS_PATH.pth
drwxrwxr-x 5 raghak raghak 4096 Oct  5 13:49 pip
drwxrwxr-x 2 raghak raghak 4096 Oct  5 13:49 pip-18.1.dist-info
drwxrwxr-x 5 raghak raghak 4096 Oct  5 13:49 pkg_resources
drwxrwxr-x 2 raghak raghak 4096 Oct  5 13:49 __pycache__
drwxrwxr-x 6 raghak raghak 4096 Oct  5 13:49 setuptools
drwxrwxr-x 2 raghak raghak 4096 Oct  5 13:49 setuptools-40.6.2.dist-info
$ cowsay-3.6.venv/bin/cowsay 'Moo!'
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

Yeah, understood. What I did is what Pants does. I'm not sure what to say then. You can try re-running like so

./pants --keep-sandboxes=on_failure

and look for a log line that tells you what the sandbox is and then cd to that sandbox and run

./__run.sh

. If that fails in the same way, can you please paste that script here?

Nothing showing up from

./pants --keep-sandboxes=on_failure

$ ./pants --keep-sandboxes=on_failure
Bootstrapping Pants using /usr/bin/python3.9
Creating the virtualenv PEX.
Downloading the Pex PEX.
SHA256 fingerprint of <https://github.com/pantsbuild/pex/releases/download/v2.1.62/pex> verified.
Installing pantsbuild.pants==2.16.0 into a virtual environment at /home/raghak/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py39
New virtual environment successfully created at /home/raghak/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py39.
11:38:00.71 [INFO] waiting for pantsd to start...
11:38:01.11 [INFO] pantsd started
11:38:01.24 [INFO] Initializing scheduler...
11:38:05.50 [INFO] Scheduler initialized.
11:38:05.52 [WARN] DEPRECATED: option 'lockfile' in scope 'flake8' is scheduled to be removed in version 2.18.0.dev0.

Custom tool versions are now installed from named resolves, as described at <https://www.pantsbuild.org/v2.16/docs/python-lockfiles>.

1. If you have an existing resolve that includes the requirements for this tool,
    you can set `[flake8].install_from_resolve = "<resolve name>".
    This may be the case if the tool also provides a runtime library, and you want
    to specify the version in just one place.
2. If not, you can set up a new resolve as described at the link above.

Either way, the resolve you choose should provide the requirements currently set by the `requirements` option for this tool, which you can see by running `pants help-advanced flake8`.
11:38:05.52 [WARN] DEPRECATED: option 'extra_requirements' in scope 'pytest' is scheduled to be removed in version 2.18.0.dev1.

Custom tool versions are now installed from named resolves, as described at <https://www.pantsbuild.org/v2.16/docs/python-lockfiles>.
11:38:05.52 [WARN] DEPRECATED: option 'lockfile' in scope 'pytest' is scheduled to be removed in version 2.18.0.dev0.

Custom tool versions are now installed from named resolves, as described at <https://www.pantsbuild.org/v2.16/docs/python-lockfiles>.

1. If you have an existing resolve that includes the requirements for this tool,
    you can set `[pytest].install_from_resolve = "<resolve name>".
    This may be the case if the tool also provides a runtime library, and you want
    to specify the version in just one place.
2. If not, you can set up a new resolve as described at the link above.

Either way, the resolve you choose should provide the requirements currently set by the `requirements` option for this tool, which you can see by running `pants help-advanced pytest`.
11:38:05.54 [WARN] Please either set `enabled = true` in the [anonymous-telemetry] section of pants.toml to enable sending anonymous stats to the Pants project to aid development, or set `enabled = false` to disable it. No telemetry sent for this run. An explicit setting will get rid of this message. See <https://www.pantsbuild.org/v2.16/docs/anonymous-telemetry> for details.
No goals specified.
Use `./pants help` to get help.
Use `./pants help goals` to list goals.

I meant to add that flag to your failing command.

I did that as well. There is no sandbox coming up in the logs. I think it is an issue with pip. I created a virtual environment with the commands

$ python3.6 -m venv env
$ source env/bin/activate
$ pip install pip --upgrade
Exception:
Traceback (most recent call last):
  File "env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "env/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 259, in run
    with self._build_session(options) as session:
  File "env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 76, in _build_session
    if options.cache_dir else None
  File "/usr/local/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not int
Traceback (most recent call last):
  File "env/bin/pip", line 11, in <module>
    sys.exit(main())
  File "env/lib/python3.6/site-packages/pip/_internal/__init__.py", line 78, in main
    return command.main(cmd_args)
  File "env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 184, in main
    timeout=min(5, options.timeout)
  File "env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 76, in _build_session
    if options.cache_dir else None
  File "/usr/local/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not int

There will be no sandbox if and only if there is no failure (on_failure). Was there a failure when you tried again this way?

There was a failure. I'll share the logs in a min

No need. Your experiment shows this has 0 to do with Pants. Agreed?

yes

Ok, so you're on your own then. You need to figure out why your Python 3.6 is broken

Thanks.. I'll check it out.

@lemon-kangaroo-17191 the obvious reading of the backtrace - which is probably wrong, but - is that you have a Pip configuration file or env var somewhere that specifies an int where it should be specifying a string.

Got it. There are no config file or env var. I'll try adding print statement and debug. Or the last way is to reinstall python3.6