Hi everyone, I’m using pants 1.17.0 and python2.7, I accidentally deleted my

.pants.d

, and when I build the pex again, I got this error:

10:54:36 00:03       [cache]                                                                                                                                                                                                          
                   No cached artifacts for 202 targets.
                   Invalidated 202 targets.**** Failed to install coverage-6.3.2 (caused by: NonZeroExit("received exit code 1 during execution of `[u'/usr/bin/python2.7', '-s', '-', 'bdist_wheel', '--dist-dir=/tmp/tmp4CEKZy']` while trying to execute `[u'/usr/bin/python2.7', '-s', '-', 'bdist_wheel', '--dist-dir=/tmp/tmp4CEKZy']`",)
):
stdout:

stderr:
Traceback (most recent call last):
  File "<stdin>", line 14, in <module>
  File "<string>", line 80
    classifier_list.append(f"Development Status :: {devstat}")
                                                            ^
SyntaxError: invalid syntax

Although I have specified the dependencies in the

BUILD

file and the version in

requirement.txt

, it still cannot build successfully. Anyone has suggestion?

f"Development Status :: {devstat}"

is python >3 code! I'm not sure where it's coming from though

Earlier in the error message it says it's trying to install

coverage-6.3.2

. So this would be in that package's

setup.py

Is that the right version in your requirements?

Coverage has dropped python 2.7 support in version 6, this same issue is likely to happen with other packages if your

requirements.txt

dependencies are not pinned

Yep I found out it tried to install

coverage-6.3.2

. That’s weird cuz I have already specified in the requirements.txt as

coverage==5.0.3

Ok, that dependency is coming from Pants trying to build its own pytest tool with coverage support. I'll give explanation 1st so you can maybe see how to find this sort of thing out and fixes last. The configuration at play here in the `[pytest]`subsystem is

cov_requirements

:https://github.com/pantsbuild/pants/blob/1.17.x/src/python/pants/backend/python/subsystems/pytest.py#L11-L22 That says give me

pytest-cov>=2.4,<2.5

. Using Pex to debug what that requirement actually means:

$ pex "pytest-cov>=2.4,<2.5" -odebug.pex
$ pex-tools debug.pex repository info --verbose --indent 4
{
    "project_name": "pytest-cov",
    "version": "2.4.0",
    "requires_python": null,
    "requires_dists": [
        "coverage>=3.7.1",
        "pytest>=2.6.0"
    ],
    "location": "/home/jsirois/.pex/installed_wheels/9b351aade9dea8ec79672d09b65476f5bb9d68e1/pytest_cov-2.4.0-py2.py3-none-any.whl"
}
{
    "project_name": "coverage",
    "version": "6.3.2",
    "requires_python": ">=3.7",
    "requires_dists": [
        "tomli; extra == \"toml\""
    ],
    "location": "/home/jsirois/.pex/installed_wheels/72ba2256c24b27cbc3753c4f55ed27435df40b1a/coverage-6.3.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
}
{
    "project_name": "pytest",
...

So you see there

pytest-cov 2.4.0

has an open-ended dependency on

coverage>=3.7.1

. It looks like

coverage 5.5

released February 28th of last year was the last version to support Python 2.7: https://pypi.org/project/coverage/5.5/ So you'll need to add a requirement to pin that low. You should probably pick the same version as in your requirements.txt. That means adding this config in `pants.ini`:

[pytest]
cov_requirements: +["coverage==5.0.3"]

I don't think that list syntax will work, note that it's a

str

option. This is old Pants before we changed it to

[pytest].version

and

[pytest].extra_requirements

. Instead...the horrible solution is that you have to replace one of the other options you're not using. Like if you don't use

unittest2

, override it to say:

[pytest]
# We don't actually use unittest2, so we
# override this to pin coverage to a version
# that still works with Python 2. This 
# is fixed in Pants 1.23+ with 
# the pytest_plugins option
unittest2_requirements = "coverage==5.0.3"

Aha, yup!

Wow! That helps! Thx so much!

Yeah - drop the `"`s so that both entries look like each other:

[pytest]
requirements: pytest==3.6.4
unittest2_requirements = coverage==5.0.3

btw, should delete the

covergae==5.0.3

in requirements.txt?

But it sometimes automatically downloads and installs the newest version of setuptools, which is 60.9.3

Yes, but you'll need to help out here. This is super old Pants and so its even more useful than normal to have detailed reports. Can you give the full command line (please add

-ldebug

to the command) and full output for an example case where this happens? The setuptools leak is probably coming from another subsystem like

[pytest]

, but without knowing what goals you're executing or where in the processing the error gets emitted, its hard to know where to start looking.

This is the log when I run

./pants test test/python:config -ldebug

This is my

pants

script. I found that there is a line specifying the setuptools version (but it’s a wrong version). Is this related to my issue?

No. That script runs just once to install pants and otherwise ~skips setup and runs installed pants. If you can run

./pants --version

, which you can by implication, the

pants

script is not the problem.

@enough-analyst-54434 Anything else I can provide?

Nope. The debug logs say enough. The failure is preparing the pytest executable that will be used to run your tests. So setuptools is being pulled in by one of these requirements:

Dumping requirement: PythonRequirement(coverage==5.0.3)
                     Dumping requirement: PythonRequirement(more-itertools<6.0.0; python_version < "3")
                     Dumping requirement: PythonRequirement(pytest-cov<2.5,>=2.4)
                     Dumping requirement: PythonRequirement(pytest-timeout<1.3,>=1.2)
                     Dumping requirement: PythonRequirement(pytest==3.6.4)

Using modern Pex to help investigate:

$ cat requirements.txt 
coverage==5.0.3
more-itertools<6.0.0; python_version < "3"
pytest-cov<2.5,>=2.4
pytest-timeout<1.3,>=1.2
pytest==3.6.4
$ pex -r requirements.txt -oresolve.pex
$ pex-tools resolve.pex repository info -vi4
{
    "project_name": "coverage",
    "version": "5.0.3",
    "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<4,>=2.7",
    "requires_dists": [
        "toml; extra == \"toml\""
    ],
    "location": "/home/jsirois/.pex/installed_wheels/ffaabb62efd5cbc2782349050b1584408b692789ab15efc702865fd340930272/coverage-5.0.3-cp310-cp310-linux_x86_64.whl"
}
{
    "project_name": "pytest-cov",
    "version": "2.4.0",
    "requires_python": null,
    "requires_dists": [
        "coverage>=3.7.1",
        "pytest>=2.6.0"
    ],
    "location": "/home/jsirois/.pex/installed_wheels/10e37e876f49ddec80d6c83a54b657157f1387ebc0f7755285f8c156130014a1/pytest_cov-2.4.0-py2.py3-none-any.whl"
}
{
    "project_name": "pytest",
    "version": "3.6.4",
    "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7",
    "requires_dists": [
        "py>=1.5.0",
        "six>=1.10.0",
        "setuptools",
        "attrs>=17.4.0",
        "more-itertools>=4.0.0",
        "atomicwrites>=1.0",
        "pluggy<0.8,>=0.5",
        "funcsigs; python_version < \"3.0\"",
        "colorama; sys_platform == \"win32\""
    ],
    "location": "/home/jsirois/.pex/installed_wheels/952c0389db115437f966c4c2079ae9d54714b9455190e56acebe14e8c38a7efa/pytest-3.6.4-py2.py3-none-any.whl"
}
...
{
    "project_name": "setuptools",
    "version": "60.10.0",
    "requires_python": ">=3.7",
...

So the issue is pytest 3.6.4 here which has an open-ended requirement of simply "setuptools". That pulls in the latest setuptools which does not work with Python 2.7.

Yep I think I can do that.

Ok. That may not be all that easy! But let us know if you run into troubles upgrading (we're deprecation happy and often flags / options / BUILD file spellings change). Generally, although painful, it's best to try bumping one version at a time and heeding the deprecation warnings.

See https://www.pantsbuild.org/v1.30/docs/upgrade-tips for upgrade tips

Thx so much! I will try upgrade now. Last time I wanted to upgrade but I was blocked by some errors, so I gave up… As long as I have your helps, I am not gonna give up this time lol

It will hopefully result in a much nicer experience! There were some big upgrades like 1.25(?) adding support for file arguments, where you can say

./pants test path/to/foo_test.py

and it will find the owning

python_tests

target & run that for you Please do let us know if you have questions on the upgrade

I got this when pip install the pantsbuild

>>> pip install pantsbuild.pants==1.30.5rc1
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at <https://pip.pypa.io/en/latest/development/release-process/#python-2-support>
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==1.30.5rc1 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, 1.17.0)
ERROR: No matching distribution found for pantsbuild.pants==1.30.5rc1

Oh I think Pants 1.16 was still using Python 2 to run Pants, later versions require upgrading the

./pants

script. But bump on our recommendation to go from 1.16 to 1.17, then 1.18 and so on. The deprecation messages should explain all these changes for you and tell how you to fix things. It is tedious to go one release at a time, but it is less confusing than trying to figure out changes like this

ok! I will do that

It went from pants.ini to pants.toml in something like 1.24. If you upgrade one release at a time, then Pants will tell you how to migrate and even has a script to mostly automate that migration

Also, you definitely shouldn't be pip installing pants! You should stick to the

pants

script, possibly refreshing it with the latest from https://pantsbuild.github.io/setup/pants as described here: https://v1.pantsbuild.org/install.html

ok I refresh the

pants

script and got this. So it looks like I should jump to 1.22.0 directly?

>>> ./pants                        
This version of the `./pants` script does not work with Pants <= 1.22.0. Instead,
either upgrade your `pants_version` or use the version of the `./pants` script at
<https://raw.githubusercontent.com/pantsbuild/setup/d1da382f6de0420940ec6007a39cba87c21075c6/pants>.

We still recommend going one release at a time. So, download https://raw.githubusercontent.com/pantsbuild/setup/d1da382f6de0420940ec6007a39cba87c21075c6/pants instead

ok~ So after I refresh the

pants

script, what should I run to check if everything is okay? Just build the pex I used to build?

yes exactly, and fix any deprecations that come up. You might want to also try running

./pants lint

and

./pants test

on a file or two

I got this error. However, that folder does not exist. What should I do?

Exception caught: (pants.build_graph.address_lookup_error.AddressLookupError) (backtrace omitted)
Exception message: Build graph construction failed: ExecutionError 1 Exception encountered:
  ResolveError: Directory "path/to/folder" does not contain any BUILD files.

This wasn't happening in a prior Pants release? It generally means that you have an address like

path/to/some/directory

or

path/to/some/directory:tgt

in a BUILD file or

pants.ini

. I recommend grepping for that path to try to find where the bad reference is

I kind of know what happened. I think there is a subdirectory that I should exclude when running pants. How do I exclude that folder?

If you're still on pants.ini, I think like this:

[GLOBAL]
pants_ignore = +["path/to/ignore_me/"]

Great! I got no dependency error now. Thank you so much! 🥺 But when I run

./pants test

, I got this. Never seen this before.

23:49:30 00:00 [main]
               (To run a reporting server: ./pants server)
23:49:31 00:01   [setup]
23:49:31 00:01     [parse]
               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> unpack-wheels -> deferred-sources -> native-compile -> link -> jvm-platform-validate -> gen -> resolve -> compile -> pyprep -> resources -> test
23:49:31 00:01   [bootstrap]
23:49:31 00:01     [substitute-aliased-targets]
23:49:31 00:01     [jar-dependency-management]
23:49:31 00:01     [bootstrap-jvm-tools]
23:49:31 00:01     [provide-tools-jar]
23:49:31 00:01   [imports]
23:49:31 00:01     [ivy-imports]
23:49:31 00:01   [unpack-jars]
23:49:31 00:01     [unpack-jars]
23:49:32 00:02   [unpack-wheels]
23:49:32 00:02     [unpack-wheels]
23:49:32 00:02   [deferred-sources]
23:49:32 00:02     [deferred-sources]
23:49:32 00:02   [native-compile]
23:49:32 00:02     [conan-prep]
23:49:32 00:02     [conan-fetch]
23:49:32 00:02     [c-for-ctypes]
23:49:32 00:02     [cpp-for-ctypes]
23:49:32 00:02   [link]
23:49:32 00:02     [shared-libraries]
23:49:32 00:02   [jvm-platform-validate]
23:49:32 00:02     [jvm-platform-validate]
23:49:32 00:02   [gen]
23:49:32 00:02     [antlr-java]
23:49:32 00:02     [antlr-py]
23:49:32 00:02     [jaxb]
23:49:32 00:02     [protoc]
23:49:32 00:02     [ragel]
23:49:32 00:02     [thrift-java]
23:49:32 00:02     [thrift-py]
23:49:32 00:02     [py-thrift-namespace-clash-check]
                   Clashing namespaces for python thrift library sources detected in build graph. This will silently
                   overwrite previously generated python sources with generated sources from thrift files declaring the
                   same python namespace. This is an upstream WONTFIX in thrift, see:
                         <https://issues.apache.org/jira/browse/THRIFT-515>
                   Errors:
                   myproject_thrift.log.session: [(c2thrift/definitions/data:gen, c2thrift/definitions/data/c2constants.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/common_log.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/diff.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/session_request_log.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/sessions.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/sessionsMobile.thrift)]
                   myproject_thrift.log.batch: [(c2thrift/definitions/data:gen, c2thrift/definitions/data/batch_logs.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/batch_request_logs.thrift)]
                   
23:49:32 00:02     [grpcio-prep]
23:49:32 00:02     [grpcio-run]
23:49:32 00:02     [wire]
23:49:32 00:02   [resolve]
23:49:32 00:02     [ivy]
23:49:32 00:02     [coursier]
23:49:32 00:02   [compile]
23:49:32 00:02     [compile-jvm-prep-command]
23:49:32 00:02       [jvm_prep_command]
23:49:32 00:02     [compile-prep-command]
23:49:32 00:02     [compile]
23:49:32 00:02     [rsc]
23:49:32 00:02     [zinc]
23:49:32 00:02     [javac]
23:49:32 00:02   [pyprep]
23:49:32 00:02     [interpreter]
23:49:32 00:02     [build-local-dists]
23:49:32 00:02     [requirements]
23:49:32 00:02     [sources]
23:49:32 00:02   [resources]
23:49:32 00:02     [prepare]
23:49:32 00:02     [services]
23:49:32 00:02   [test]
23:49:32 00:02     [test-jvm-prep-command]
23:49:32 00:02       [jvm_prep_command]
23:49:32 00:02     [test-prep-command]
23:49:32 00:02     [legacy]
23:49:32 00:02     [pytest-prep]
23:49:33 00:03     [pytest]
23:49:33 00:03       [cache] 
                   No cached artifacts for 1 target.
                   Invalidated 1 target.
23:49:33 00:03       [run]
                     /home/cedriczheng/myrepo/.pants.d/test/pytest-prep/CPython-2.7.12/0ef27f900708c3ec08d3ddb7bff04c4eaa061e2c/.deps/setuptools-46.4.0-py2-none-any.whl/pkg_resources/py2_warn.py:21: UserWarning: Setuptools will stop working on Python 2
                     ************************************************************
                     You are running Setuptools on Python 2, which is no longer
                     supported and
                     >>> SETUPTOOLS WILL STOP WORKING <<<
                     in a subsequent release (no sooner than 2020-04-20).
                     Please ensure you are installing
                     Setuptools using pip 9.x or later or pin to `setuptools<45`
                     in your environment.
                     If you have done those things and are still encountering
                     this message, please follow up at
                     <https://bit.ly/setuptools-py2-warning>.
                     ************************************************************
                       sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
                     ============== test session starts ===============
                     platform linux2 -- Python 2.7.12, pytest-3.6.4, py-1.11.0, pluggy-0.7.1
                     rootdir: /home/cedriczheng/myrepo, inifile: /dev/null
                     plugins: cov-2.4.0, timeout-1.2.1
                     collected 42 items
                     
                     test/python/myproject_test/log/config/current_config_test.py E [  2%]
                     
                     
                     ===================== ERRORS =====================
                     _ ERROR at setup of CurrentConfigTestAws.test_adwords_config _
                     
                     cls = <class 'myproject_test.log.config.current_config_test.CurrentConfigTestAws'>
                     
                         @classmethod
                         def setUpClass(cls):
                             cluster().name = cls.cluster_name
                         
                             cls.sources_mock = mock.patch('myproject.log.config.parser.parser._config_sources')
                             sources = cls.sources_mock.start()
                             sources.return_value = [DirConfigSource('config')]
                     >       load_processors()
                     
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject_test/log/config/current_config_test.py:23: 
                     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject/log/config/tables/__init__.py:23: in load_processors
                         load_processors([submodule])
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject/log/config/tables/__init__.py:21: in load_processors
                         submodule = importlib.import_module(name)
                     /usr/lib/python2.7/importlib/__init__.py:37: in import_module
                         __import__(name)
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject/streaming/sfdc/sfdc_download.py:10: in <module>
                         from salesforce_bulk.util import IteratorBytesIO
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/salesforce_bulk-2.0.0-py2.py3-none-any.whl/salesforce_bulk/__init__.py:2: in <module>
                         from .salesforce_bulk import SalesforceBulk, BulkApiError, UploadResult
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/salesforce_bulk-2.0.0-py2.py3-none-any.whl/salesforce_bulk/salesforce_bulk.py:22: in <module>
                         from simple_salesforce import SalesforceLogin
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/simple_salesforce-0.75.3-py2.py3-none-any.whl/simple_salesforce/__init__.py:4: in <module>
                         from simple_salesforce.api import (
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/simple_salesforce-0.75.3-py2.py3-none-any.whl/simple_salesforce/api.py:21: in <module>
                         from simple_salesforce.login import SalesforceLogin
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/simple_salesforce-0.75.3-py2.py3-none-any.whl/simple_salesforce/login.py:24: in <module>
                         from authlib.jose import jwt
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/__init__.py:11: in <module>
                         from .rfc7516 import (
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/rfc7516/__init__.py:11: in <module>
                         from .jwe import JsonWebEncryption
                     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                     
                         from collections import OrderedDict
                         from copy import deepcopy
                         
                         from authlib.common.encoding import (
                             to_bytes, urlsafe_b64encode, json_b64encode, to_unicode
                         )
                     >   from authlib.jose.rfc7516.models import JWEAlgorithmWithTagAwareKeyAgreement, JWESharedHeader, JWEHeader
                     E     File "/home/cedriczheng/myrepo/.pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/rfc7516/models.py", line 5
                     E       class JWEAlgorithmBase(object, metaclass=ABCMeta):
                     E                                               ^
                     E   SyntaxError: invalid syntax
                     
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/rfc7516/jwe.py:7: SyntaxError
                     - generated xml file: /home/cedriczheng/myrepo/.pants.d/test/pytest/test.python.myproject_test.log.config.current_config_test/junitxml/TEST-test.python.myproject_test.log.config.current_config_test.xml -
                     ============ 1 error in 2.66 seconds =============
                     
                   test/python/myproject_test/log/config:current_config_test                           .....   FAILURE
FAILURE: FAILURE


               Waiting for background workers to finish.
23:49:37 00:07   [complete]
               FAILURE

But when I run ./pants test , I got this.

I'm heading to bed, but I'm confused. Are you getting the test issue on Pants 1.17, which you were already using?

o I modified the version to 1.18.0 in pants.ini and run ./pants --version again and then got this.

Did you update the

./pants

script to https://raw.githubusercontent.com/pantsbuild/setup/d1da382f6de0420940ec6007a39cba87c21075c6/pants?

Oh damn, I was switching between branch and it turns out I didn’t refresh the

./pants

script. Sorry for bothering! Now the

./pants --version

works fine. Thx! Good night!