https://pantsbuild.org/ logo
#general
Title
# general
f

freezing-area-97131

03/10/2022, 11:03 AM
Hi everyone, I’m using pants 1.17.0 and python2.7, I accidentally deleted my
.pants.d
, and when I build the pex again, I got this error:
Copy code
10:54:36 00:03       [cache]                                                                                                                                                                                                          
                   No cached artifacts for 202 targets.
                   Invalidated 202 targets.**** Failed to install coverage-6.3.2 (caused by: NonZeroExit("received exit code 1 during execution of `[u'/usr/bin/python2.7', '-s', '-', 'bdist_wheel', '--dist-dir=/tmp/tmp4CEKZy']` while trying to execute `[u'/usr/bin/python2.7', '-s', '-', 'bdist_wheel', '--dist-dir=/tmp/tmp4CEKZy']`",)
):
stdout:

stderr:
Traceback (most recent call last):
  File "<stdin>", line 14, in <module>
  File "<string>", line 80
    classifier_list.append(f"Development Status :: {devstat}")
                                                            ^
SyntaxError: invalid syntax
Although I have specified the dependencies in the
BUILD
file and the version in
requirement.txt
, it still cannot build successfully. Anyone has suggestion?
p

plain-fireman-49959

03/10/2022, 11:59 AM
f"Development Status :: {devstat}"
is python >3 code! I'm not sure where it's coming from though
b

bitter-ability-32190

03/10/2022, 12:07 PM
Earlier in the error message it says it's trying to install
coverage-6.3.2
. So this would be in that package's
setup.py
Is that the right version in your requirements?
p

plain-fireman-49959

03/10/2022, 12:28 PM
Coverage has dropped python 2.7 support in version 6, this same issue is likely to happen with other packages if your
requirements.txt
dependencies are not pinned
Would you mind sharing (part of) the BUILD file and requirements.txt?
f

freezing-area-97131

03/10/2022, 12:45 PM
Yep I found out it tried to install
coverage-6.3.2
. That’s weird cuz I have already specified in the requirements.txt as
coverage==5.0.3
This is my requirementx.txt
Copy code
beaker==1.6.4
bingads==13.0.11
cachetools==3.1.1
cassandra-driver==3.6.0
clint==0.5.1
cm-api==14.0.0
contextlib2==0.5.5
coverage==5.0.3
cryptography==3.2.1
decorator==4.4.2
docutils==0.14
facebook_business==12.0.0
freezegun==0.3.14
functools32==3.2.3-1
google-api-python-client==1.6.2
google-auth==1.30.0
gunicorn==19.7.1
httplib2==0.9.2
humanize==0.5.1
hypothesis==4.26.3
inflection==0.3.1
jenkinsapi==0.3.9
mock==2.0.0
mockredispy==2.9.3
jinja2==2.11.0
markupsafe==1.1.1
numpy==1.14.1
click==7.1.2
oauth2client==4.0.0
p4python==2015.2.1348262
phonenumbers==7.5.1
pymysql==0.7.9
pystache==0.5.4
python-dateutil==2.6.1
pyzmq==19.0.2
redis>=2.10.6,<=3.5.3
redlock==1.2.0
setuptools==46.4.0
simple-salesforce==0.75.3
six==1.15.0
sqlparse==0.3.1
sumologic-sdk==0.1.7
suds-jurko==0.6
tabulate==0.8.9
thrift-sasl==0.3.0
thrift==0.9.3
twilio==5.4.0
ujson==1.35
voluptuous==0.9.3
xlrd==1.1.0
itsdangerous==1.1.0
boto==2.49.0
sh==1.12.14
WTForms==2.3.3
google-crc32c==1.1.0
google-resumable-media==0.3.1
And in
BUILD
, I specified a few dependencies like this
Copy code
python_requirement_library(
    name='gevent',
    dependencies=[
        ':setuptools',
        ':coverage',
    ],
)
e

enough-analyst-54434

03/10/2022, 1:04 PM
Ok, that dependency is coming from Pants trying to build its own pytest tool with coverage support. I'll give explanation 1st so you can maybe see how to find this sort of thing out and fixes last. The configuration at play here in the `[pytest]`subsystem is
cov_requirements
:https://github.com/pantsbuild/pants/blob/1.17.x/src/python/pants/backend/python/subsystems/pytest.py#L11-L22 That says give me
pytest-cov>=2.4,<2.5
. Using Pex to debug what that requirement actually means:
Copy code
$ pex "pytest-cov>=2.4,<2.5" -odebug.pex
$ pex-tools debug.pex repository info --verbose --indent 4
{
    "project_name": "pytest-cov",
    "version": "2.4.0",
    "requires_python": null,
    "requires_dists": [
        "coverage>=3.7.1",
        "pytest>=2.6.0"
    ],
    "location": "/home/jsirois/.pex/installed_wheels/9b351aade9dea8ec79672d09b65476f5bb9d68e1/pytest_cov-2.4.0-py2.py3-none-any.whl"
}
{
    "project_name": "coverage",
    "version": "6.3.2",
    "requires_python": ">=3.7",
    "requires_dists": [
        "tomli; extra == \"toml\""
    ],
    "location": "/home/jsirois/.pex/installed_wheels/72ba2256c24b27cbc3753c4f55ed27435df40b1a/coverage-6.3.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
}
{
    "project_name": "pytest",
...
So you see there
pytest-cov 2.4.0
has an open-ended dependency on
coverage>=3.7.1
. It looks like
coverage 5.5
released February 28th of last year was the last version to support Python 2.7: https://pypi.org/project/coverage/5.5/ So you'll need to add a requirement to pin that low. You should probably pick the same version as in your requirements.txt. That means adding this config in `pants.ini`:
Copy code
[pytest]
cov_requirements: +["coverage==5.0.3"]
h

hundreds-father-404

03/10/2022, 5:51 PM
I don't think that list syntax will work, note that it's a
str
option. This is old Pants before we changed it to
[pytest].version
and
[pytest].extra_requirements
. Instead...the horrible solution is that you have to replace one of the other options you're not using. Like if you don't use
unittest2
, override it to say:
Copy code
[pytest]
# We don't actually use unittest2, so we
# override this to pin coverage to a version
# that still works with Python 2. This 
# is fixed in Pants 1.23+ with 
# the pytest_plugins option
unittest2_requirements = "coverage==5.0.3"
e

enough-analyst-54434

03/10/2022, 5:55 PM
Aha, yup!
f

freezing-area-97131

03/10/2022, 10:44 PM
Wow! That helps! Thx so much!
But then I encountered this:
Copy code
22:42:06 00:07   [complete]
               FAILURE
timestamp: 2022-03-10T22:42:06.256931
Exception caught: (pkg_resources.RequirementParseError) (backtrace omitted)
Exception message: Invalid requirement, parse error at "u'"coverag'"
My
pants.ini
is like this:
Copy code
[pytest]
requirements: pytest==3.6.4
unittest2_requirements = "coverage==5.0.3"
e

enough-analyst-54434

03/10/2022, 10:48 PM
Yeah - drop the `"`s so that both entries look like each other:
Copy code
[pytest]
requirements: pytest==3.6.4
unittest2_requirements = coverage==5.0.3
f

freezing-area-97131

03/10/2022, 10:52 PM
btw, should delete the
covergae==5.0.3
in requirements.txt?
oh I think it passed the blocker now!
❤️ 1
I got one more question. Though I have already bypassed it, it may happen again in the future. I have specified the version of
setuptools
in
requirements.txt
like this:
Copy code
setuptools==46.4.0
But it sometimes automatically downloads and installs the newest version of setuptools, which is 60.9.3, and I have to remove it manually. Where can I pin the version?
@enough-analyst-54434 @hundreds-father-404 Can you also take a look when you got time? I really appreciate your help. 🥺
e

enough-analyst-54434

03/11/2022, 11:25 PM
But it sometimes automatically downloads and installs the newest version of setuptools, which is 60.9.3
Yes, but you'll need to help out here. This is super old Pants and so its even more useful than normal to have detailed reports. Can you give the full command line (please add
-ldebug
to the command) and full output for an example case where this happens? The setuptools leak is probably coming from another subsystem like
[pytest]
, but without knowing what goals you're executing or where in the processing the error gets emitted, its hard to know where to start looking.
👍 2
f

freezing-area-97131

03/17/2022, 12:16 AM
This is the log when I run
./pants test test/python:config -ldebug
This is my
pants
script. I found that there is a line specifying the setuptools version (but it’s a wrong version). Is this related to my issue?
Copy code
#!/usr/bin/env bash
# Copyright 2015 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).

# =============================== NOTE ===============================
# This pants bootstrap script comes from the pantsbuild/setup
# project and is intended to be checked into your code repository so
# that any developer can check out your code and be building it with
# pants with no prior setup needed.
#
# You can learn more here: <https://pantsbuild.github.io/setup>
# ====================================================================

set -e

PYTHON=${PYTHON:-$(which python2.7)}

PANTS_HOME="${PANTS_HOME:-${XDG_CACHE_HOME:-$HOME/.cache}/pants/setup}"
PANTS_BOOTSTRAP="${PANTS_HOME}/bootstrap-$(uname -s)-$(uname -m)"

VENV_VERSION=${VENV_VERSION:-15.2.0}

# This requirement is installed before pantsbuild.pants to hack around
# interpreters that have newer setuptools already installed, effectively
# re-installing an older setuptools and pinning low to a version known to be 
# ingestable by both pants and pex for all reasonable versions of pants.
# See:
#   <https://github.com/pantsbuild/pants/issues/3948>
#   <https://github.com/pantsbuild/setup/issues/14>
#   <https://github.com/pantsbuild/setup/issues/19>
SETUPTOOLS_REQUIREMENT="setuptools==5.4.1"

VENV_PACKAGE=virtualenv-${VENV_VERSION}
VENV_TARBALL=${VENV_PACKAGE}.tar.gz

# The high-level flow:
# 1.) Grab pants version from pants.ini or default to latest.
# 2.) Check for a venv via a naming/path convention and execute if found.
# 3.) Otherwise create venv and re-exec self.
#
# After that pants itself will handle making sure any requested plugins
# are installed and up to date.

function tempdir {
  mktemp -d "$1"/pants.XXXXXX
}

# TODO(John Sirois): GC race loser tmp dirs leftover from bootstrap_XXX
# functions.  Any tmp dir w/o a symlink pointing to it can go.

function bootstrap_venv {
  if [[ ! -d "${PANTS_BOOTSTRAP}/${VENV_PACKAGE}" ]]
  then
    (
      mkdir -p "${PANTS_BOOTSTRAP}" && \
      staging_dir=$(tempdir "${PANTS_BOOTSTRAP}") && \
      cd "${staging_dir}" && \
      curl -LO <https://pypi.io/packages/source/v/virtualenv/${VENV_TARBALL}> && \
      tar -xzf ${VENV_TARBALL} && \
      ln -s "${staging_dir}/${VENV_PACKAGE}" "${staging_dir}/latest" && \
      mv "${staging_dir}/latest" "${PANTS_BOOTSTRAP}/${VENV_PACKAGE}"
    ) 1>&2
  fi
  echo "${PANTS_BOOTSTRAP}/${VENV_PACKAGE}"
}

function bootstrap_pants {
  pants_requirement="pantsbuild.pants"
  pants_version=$(
    grep -E "^[[:space:]]*pants_version" pants.ini 2>/dev/null | \
      cut -f2 -d: | tr -d " "
  )
  if [[ -n "${pants_version}" ]]
  then
    pants_requirement="${pants_requirement}==${pants_version}"
  else
    pants_version="unspecified"
  fi

  if [[ ! -d "${PANTS_BOOTSTRAP}/${pants_version}" ]]
  then
    (
      # NB: We setup the virtualenv with no setuptools to ensure our
      # ${SETUPTOOLS_REQUIREMENT} wins.
      venv_path="$(bootstrap_venv)" && \
      staging_dir=$(tempdir "${PANTS_BOOTSTRAP}") && \
      "${PYTHON}" "${venv_path}/virtualenv.py" --no-setuptools --no-download \
        "${staging_dir}/install" && \
      "${staging_dir}/install/bin/python" \
        "${staging_dir}/install/bin/pip" install \
          "${SETUPTOOLS_REQUIREMENT}" && \
      "${staging_dir}/install/bin/python" \
        "${staging_dir}/install/bin/pip" install \
          "${pants_requirement}" && \
      ln -s "${staging_dir}/install" "${staging_dir}/${pants_version}" && \
      mv "${staging_dir}/${pants_version}" "${PANTS_BOOTSTRAP}/${pants_version}"
    ) 1>&2
  fi
  echo "${PANTS_BOOTSTRAP}/${pants_version}"
}

pants_dir=$(bootstrap_pants) && \
exec "${pants_dir}/bin/python" "${pants_dir}/bin/pants" "$@"
Also my
pants.ini
Copy code
[GLOBAL]
pants_version: 1.17.0
build_file_imports: allow
binaries_path_by_id: {('darwin', '18'): ('mac', '10.13')}
fail_fast: True

[thrift]
version: 0.9.3

[pytest]
requirements: pytest==3.6.4
unittest2_requirements = coverage==5.0.3

[ivy]
ivy_settings: %(pants_supportdir)s/ivy/ivysettings.xml

[scala]
version: 2.10

[jvm-distributions]
paths: {
    'macos': [
      '/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk',
      '/Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk',
    ],
    'linux': [
      '/usr/lib/jvm/java-8-openjdk-amd64',
      '/usr/lib/jvm/java-7-openjdk-amd64',
    ],
  }

[jvm-platform]
default_platform: java7
platforms: {
    'java7': {'source': '7', 'target': '7', 'args': [] },
    'java8': {'source': '8', 'target': '8', 'args': [] },
  }
@enough-analyst-54434 Anything else I can provide?
e

enough-analyst-54434

03/17/2022, 12:33 AM
This is my
pants
script. I found that there is a line specifying the setuptools version (but it’s a wrong version). Is this related to my issue?
No. That script runs just once to install pants and otherwise ~skips setup and runs installed pants. If you can run
./pants --version
, which you can by implication, the
pants
script is not the problem.
@enough-analyst-54434 Anything else I can provide?
Nope. The debug logs say enough. The failure is preparing the pytest executable that will be used to run your tests. So setuptools is being pulled in by one of these requirements:
Copy code
Dumping requirement: PythonRequirement(coverage==5.0.3)
                     Dumping requirement: PythonRequirement(more-itertools<6.0.0; python_version < "3")
                     Dumping requirement: PythonRequirement(pytest-cov<2.5,>=2.4)
                     Dumping requirement: PythonRequirement(pytest-timeout<1.3,>=1.2)
                     Dumping requirement: PythonRequirement(pytest==3.6.4)
Using modern Pex to help investigate:
Copy code
$ cat requirements.txt 
coverage==5.0.3
more-itertools<6.0.0; python_version < "3"
pytest-cov<2.5,>=2.4
pytest-timeout<1.3,>=1.2
pytest==3.6.4
$ pex -r requirements.txt -oresolve.pex
$ pex-tools resolve.pex repository info -vi4
{
    "project_name": "coverage",
    "version": "5.0.3",
    "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<4,>=2.7",
    "requires_dists": [
        "toml; extra == \"toml\""
    ],
    "location": "/home/jsirois/.pex/installed_wheels/ffaabb62efd5cbc2782349050b1584408b692789ab15efc702865fd340930272/coverage-5.0.3-cp310-cp310-linux_x86_64.whl"
}
{
    "project_name": "pytest-cov",
    "version": "2.4.0",
    "requires_python": null,
    "requires_dists": [
        "coverage>=3.7.1",
        "pytest>=2.6.0"
    ],
    "location": "/home/jsirois/.pex/installed_wheels/10e37e876f49ddec80d6c83a54b657157f1387ebc0f7755285f8c156130014a1/pytest_cov-2.4.0-py2.py3-none-any.whl"
}
{
    "project_name": "pytest",
    "version": "3.6.4",
    "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7",
    "requires_dists": [
        "py>=1.5.0",
        "six>=1.10.0",
        "setuptools",
        "attrs>=17.4.0",
        "more-itertools>=4.0.0",
        "atomicwrites>=1.0",
        "pluggy<0.8,>=0.5",
        "funcsigs; python_version < \"3.0\"",
        "colorama; sys_platform == \"win32\""
    ],
    "location": "/home/jsirois/.pex/installed_wheels/952c0389db115437f966c4c2079ae9d54714b9455190e56acebe14e8c38a7efa/pytest-3.6.4-py2.py3-none-any.whl"
}
...
{
    "project_name": "setuptools",
    "version": "60.10.0",
    "requires_python": ">=3.7",
...
So the issue is pytest 3.6.4 here which has an open-ended requirement of simply "setuptools". That pulls in the latest setuptools which does not work with Python 2.7.
So @freezing-area-97131 we might be able to work around this, but the problems won't end there. The underlying problem is this Pants version is old enough it uses an ancient version of Pex that has its own resolver that is pretty broken (modern Pex uses Pip). That old Pex resolver is not respecting
Requires-Python
metadata; so it's picking distributions that do not work with Python 2.7, like modern setuptools which says
"requires_python": ">=3.7",
Your best bet is to upgrade to the latest Pants v1 which is https://pypi.org/project/pantsbuild.pants/1.30.5rc1/ That uses Pex 2.1.24 which handles this sort of thing.
@freezing-area-97131 are you in a position to do that?
f

freezing-area-97131

03/17/2022, 12:38 AM
Yep I think I can do that.
e

enough-analyst-54434

03/17/2022, 12:39 AM
Ok. That may not be all that easy! But let us know if you run into troubles upgrading (we're deprecation happy and often flags / options / BUILD file spellings change). Generally, although painful, it's best to try bumping one version at a time and heeding the deprecation warnings.
👍 1
h

hundreds-father-404

03/17/2022, 12:44 AM
f

freezing-area-97131

03/17/2022, 12:46 AM
Thx so much! I will try upgrade now. Last time I wanted to upgrade but I was blocked by some errors, so I gave up… As long as I have your helps, I am not gonna give up this time lol
❤️ 1
h

hundreds-father-404

03/17/2022, 12:48 AM
It will hopefully result in a much nicer experience! There were some big upgrades like 1.25(?) adding support for file arguments, where you can say
./pants test path/to/foo_test.py
and it will find the owning
python_tests
target & run that for you Please do let us know if you have questions on the upgrade
f

freezing-area-97131

03/17/2022, 12:55 AM
I got this when pip install the pantsbuild
Copy code
>>> pip install pantsbuild.pants==1.30.5rc1
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at <https://pip.pypa.io/en/latest/development/release-process/#python-2-support>
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==1.30.5rc1 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, 1.17.0)
ERROR: No matching distribution found for pantsbuild.pants==1.30.5rc1
@hundreds-father-404 hmm I don’t have a
pants.toml
h

hundreds-father-404

03/17/2022, 12:58 AM
Oh I think Pants 1.16 was still using Python 2 to run Pants, later versions require upgrading the
./pants
script. But bump on our recommendation to go from 1.16 to 1.17, then 1.18 and so on. The deprecation messages should explain all these changes for you and tell how you to fix things. It is tedious to go one release at a time, but it is less confusing than trying to figure out changes like this
f

freezing-area-97131

03/17/2022, 12:58 AM
ok! I will do that
h

hundreds-father-404

03/17/2022, 12:58 AM
It went from pants.ini to pants.toml in something like 1.24. If you upgrade one release at a time, then Pants will tell you how to migrate and even has a script to mostly automate that migration
e

enough-analyst-54434

03/17/2022, 1:58 AM
Also, you definitely shouldn't be pip installing pants! You should stick to the
pants
script, possibly refreshing it with the latest from https://pantsbuild.github.io/setup/pants as described here: https://v1.pantsbuild.org/install.html
1
f

freezing-area-97131

03/17/2022, 2:08 AM
ok I refresh the
pants
script and got this. So it looks like I should jump to 1.22.0 directly?
Copy code
>>> ./pants                        
This version of the `./pants` script does not work with Pants <= 1.22.0. Instead,
either upgrade your `pants_version` or use the version of the `./pants` script at
<https://raw.githubusercontent.com/pantsbuild/setup/d1da382f6de0420940ec6007a39cba87c21075c6/pants>.
Or 1.23.0
h

hundreds-father-404

03/17/2022, 2:10 AM
We still recommend going one release at a time. So, download https://raw.githubusercontent.com/pantsbuild/setup/d1da382f6de0420940ec6007a39cba87c21075c6/pants instead
f

freezing-area-97131

03/17/2022, 2:16 AM
ok~ So after I refresh the
pants
script, what should I run to check if everything is okay? Just build the pex I used to build?
These?
Copy code
$ ./pants 
$ ./pants list :: > /dev/null
$ ./pants filedeps :: > /dev/null
h

hundreds-father-404

03/17/2022, 2:30 AM
yes exactly, and fix any deprecations that come up. You might want to also try running
./pants lint
and
./pants test
on a file or two
and then I suggest you land the upgrade in your CI every couple of versions. For example, locally go from 1.16->1.17->1.18->1.19, then put up a PR to upgrade to 1.19 and actually land it. This suggestion is so that it's easier for you to catch any issues that come up, and to revert if you need to
f

freezing-area-97131

03/17/2022, 5:04 AM
I got this error. However, that folder does not exist. What should I do?
Copy code
Exception caught: (pants.build_graph.address_lookup_error.AddressLookupError) (backtrace omitted)
Exception message: Build graph construction failed: ExecutionError 1 Exception encountered:
  ResolveError: Directory "path/to/folder" does not contain any BUILD files.
h

hundreds-father-404

03/17/2022, 5:08 AM
This wasn't happening in a prior Pants release? It generally means that you have an address like
path/to/some/directory
or
path/to/some/directory:tgt
in a BUILD file or
pants.ini
. I recommend grepping for that path to try to find where the bad reference is
f

freezing-area-97131

03/17/2022, 5:10 AM
I kind of know what happened. I think there is a subdirectory that I should exclude when running pants. How do I exclude that folder?
h

hundreds-father-404

03/17/2022, 5:15 AM
If you're still on pants.ini, I think like this:
Copy code
[GLOBAL]
pants_ignore = +["path/to/ignore_me/"]
f

freezing-area-97131

03/17/2022, 6:52 AM
Great! I got no dependency error now. Thank you so much! 🥺 But when I run
./pants test
, I got this. Never seen this before.
Copy code
23:49:30 00:00 [main]
               (To run a reporting server: ./pants server)
23:49:31 00:01   [setup]
23:49:31 00:01     [parse]
               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> unpack-wheels -> deferred-sources -> native-compile -> link -> jvm-platform-validate -> gen -> resolve -> compile -> pyprep -> resources -> test
23:49:31 00:01   [bootstrap]
23:49:31 00:01     [substitute-aliased-targets]
23:49:31 00:01     [jar-dependency-management]
23:49:31 00:01     [bootstrap-jvm-tools]
23:49:31 00:01     [provide-tools-jar]
23:49:31 00:01   [imports]
23:49:31 00:01     [ivy-imports]
23:49:31 00:01   [unpack-jars]
23:49:31 00:01     [unpack-jars]
23:49:32 00:02   [unpack-wheels]
23:49:32 00:02     [unpack-wheels]
23:49:32 00:02   [deferred-sources]
23:49:32 00:02     [deferred-sources]
23:49:32 00:02   [native-compile]
23:49:32 00:02     [conan-prep]
23:49:32 00:02     [conan-fetch]
23:49:32 00:02     [c-for-ctypes]
23:49:32 00:02     [cpp-for-ctypes]
23:49:32 00:02   [link]
23:49:32 00:02     [shared-libraries]
23:49:32 00:02   [jvm-platform-validate]
23:49:32 00:02     [jvm-platform-validate]
23:49:32 00:02   [gen]
23:49:32 00:02     [antlr-java]
23:49:32 00:02     [antlr-py]
23:49:32 00:02     [jaxb]
23:49:32 00:02     [protoc]
23:49:32 00:02     [ragel]
23:49:32 00:02     [thrift-java]
23:49:32 00:02     [thrift-py]
23:49:32 00:02     [py-thrift-namespace-clash-check]
                   Clashing namespaces for python thrift library sources detected in build graph. This will silently
                   overwrite previously generated python sources with generated sources from thrift files declaring the
                   same python namespace. This is an upstream WONTFIX in thrift, see:
                         <https://issues.apache.org/jira/browse/THRIFT-515>
                   Errors:
                   myproject_thrift.log.session: [(c2thrift/definitions/data:gen, c2thrift/definitions/data/c2constants.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/common_log.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/diff.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/session_request_log.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/sessions.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/sessionsMobile.thrift)]
                   myproject_thrift.log.batch: [(c2thrift/definitions/data:gen, c2thrift/definitions/data/batch_logs.thrift), (c2thrift/definitions/data:gen, c2thrift/definitions/data/batch_request_logs.thrift)]
                   
23:49:32 00:02     [grpcio-prep]
23:49:32 00:02     [grpcio-run]
23:49:32 00:02     [wire]
23:49:32 00:02   [resolve]
23:49:32 00:02     [ivy]
23:49:32 00:02     [coursier]
23:49:32 00:02   [compile]
23:49:32 00:02     [compile-jvm-prep-command]
23:49:32 00:02       [jvm_prep_command]
23:49:32 00:02     [compile-prep-command]
23:49:32 00:02     [compile]
23:49:32 00:02     [rsc]
23:49:32 00:02     [zinc]
23:49:32 00:02     [javac]
23:49:32 00:02   [pyprep]
23:49:32 00:02     [interpreter]
23:49:32 00:02     [build-local-dists]
23:49:32 00:02     [requirements]
23:49:32 00:02     [sources]
23:49:32 00:02   [resources]
23:49:32 00:02     [prepare]
23:49:32 00:02     [services]
23:49:32 00:02   [test]
23:49:32 00:02     [test-jvm-prep-command]
23:49:32 00:02       [jvm_prep_command]
23:49:32 00:02     [test-prep-command]
23:49:32 00:02     [legacy]
23:49:32 00:02     [pytest-prep]
23:49:33 00:03     [pytest]
23:49:33 00:03       [cache] 
                   No cached artifacts for 1 target.
                   Invalidated 1 target.
23:49:33 00:03       [run]
                     /home/cedriczheng/myrepo/.pants.d/test/pytest-prep/CPython-2.7.12/0ef27f900708c3ec08d3ddb7bff04c4eaa061e2c/.deps/setuptools-46.4.0-py2-none-any.whl/pkg_resources/py2_warn.py:21: UserWarning: Setuptools will stop working on Python 2
                     ************************************************************
                     You are running Setuptools on Python 2, which is no longer
                     supported and
                     >>> SETUPTOOLS WILL STOP WORKING <<<
                     in a subsequent release (no sooner than 2020-04-20).
                     Please ensure you are installing
                     Setuptools using pip 9.x or later or pin to `setuptools<45`
                     in your environment.
                     If you have done those things and are still encountering
                     this message, please follow up at
                     <https://bit.ly/setuptools-py2-warning>.
                     ************************************************************
                       sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
                     ============== test session starts ===============
                     platform linux2 -- Python 2.7.12, pytest-3.6.4, py-1.11.0, pluggy-0.7.1
                     rootdir: /home/cedriczheng/myrepo, inifile: /dev/null
                     plugins: cov-2.4.0, timeout-1.2.1
                     collected 42 items
                     
                     test/python/myproject_test/log/config/current_config_test.py E [  2%]
                     
                     
                     ===================== ERRORS =====================
                     _ ERROR at setup of CurrentConfigTestAws.test_adwords_config _
                     
                     cls = <class 'myproject_test.log.config.current_config_test.CurrentConfigTestAws'>
                     
                         @classmethod
                         def setUpClass(cls):
                             cluster().name = cls.cluster_name
                         
                             cls.sources_mock = mock.patch('myproject.log.config.parser.parser._config_sources')
                             sources = cls.sources_mock.start()
                             sources.return_value = [DirConfigSource('config')]
                     >       load_processors()
                     
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject_test/log/config/current_config_test.py:23: 
                     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject/log/config/tables/__init__.py:23: in load_processors
                         load_processors([submodule])
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject/log/config/tables/__init__.py:21: in load_processors
                         submodule = importlib.import_module(name)
                     /usr/lib/python2.7/importlib/__init__.py:37: in import_module
                         __import__(name)
                     .pants.d/pyprep/sources/1d7a2f20242bd78d9edb42910040da9e0a7d1c42/myproject/streaming/sfdc/sfdc_download.py:10: in <module>
                         from salesforce_bulk.util import IteratorBytesIO
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/salesforce_bulk-2.0.0-py2.py3-none-any.whl/salesforce_bulk/__init__.py:2: in <module>
                         from .salesforce_bulk import SalesforceBulk, BulkApiError, UploadResult
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/salesforce_bulk-2.0.0-py2.py3-none-any.whl/salesforce_bulk/salesforce_bulk.py:22: in <module>
                         from simple_salesforce import SalesforceLogin
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/simple_salesforce-0.75.3-py2.py3-none-any.whl/simple_salesforce/__init__.py:4: in <module>
                         from simple_salesforce.api import (
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/simple_salesforce-0.75.3-py2.py3-none-any.whl/simple_salesforce/api.py:21: in <module>
                         from simple_salesforce.login import SalesforceLogin
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/simple_salesforce-0.75.3-py2.py3-none-any.whl/simple_salesforce/login.py:24: in <module>
                         from authlib.jose import jwt
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/__init__.py:11: in <module>
                         from .rfc7516 import (
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/rfc7516/__init__.py:11: in <module>
                         from .jwe import JsonWebEncryption
                     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                     
                         from collections import OrderedDict
                         from copy import deepcopy
                         
                         from authlib.common.encoding import (
                             to_bytes, urlsafe_b64encode, json_b64encode, to_unicode
                         )
                     >   from authlib.jose.rfc7516.models import JWEAlgorithmWithTagAwareKeyAgreement, JWESharedHeader, JWEHeader
                     E     File "/home/cedriczheng/myrepo/.pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/rfc7516/models.py", line 5
                     E       class JWEAlgorithmBase(object, metaclass=ABCMeta):
                     E                                               ^
                     E   SyntaxError: invalid syntax
                     
                     .pants.d/pyprep/requirements/CPython-2.7.12/086f151e9f85167ea1c8271003c0d44c398697af/.deps/Authlib-1.0.0-py2.py3-none-any.whl/authlib/jose/rfc7516/jwe.py:7: SyntaxError
                     - generated xml file: /home/cedriczheng/myrepo/.pants.d/test/pytest/test.python.myproject_test.log.config.current_config_test/junitxml/TEST-test.python.myproject_test.log.config.current_config_test.xml -
                     ============ 1 error in 2.66 seconds =============
                     
                   test/python/myproject_test/log/config:current_config_test                           .....   FAILURE
FAILURE: FAILURE


               Waiting for background workers to finish.
23:49:37 00:07   [complete]
               FAILURE
oh btw, I found out I’m still in 1.17.0 , so I modified the version to 1.18.0 in
pants.ini
and run
./pants --version
again and then got this.
Copy code
>>> ./pants --version
New python executable in /home/cedriczheng/.cache/pants/setup/bootstrap-Linux-x86_64/pants.JTDE2a/install/bin/python2.7
Also creating executable in /home/cedriczheng/.cache/pants/setup/bootstrap-Linux-x86_64/pants.JTDE2a/install/bin/python
Installing pip, wheel...done.
Collecting setuptools==5.4.1
  Using cached <https://files.pythonhosted.org/packages/b1/ba/02218786fe9d5beeb2cfdaf0e423219d132845774d703f6e8a708dd5e642/setuptools-5.4.1-py2.py3-none-any.whl>
Installing collected packages: setuptools
Successfully installed setuptools-5.4.1
You are using pip version 9.0.3, however version 22.0.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pantsbuild.pants==1.18.0
  Could not find a version that satisfies the requirement pantsbuild.pants==1.18.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, 1.17.0)
No matching distribution found for pantsbuild.pants==1.18.0
You are using pip version 9.0.3, however version 22.0.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
./pants: line 103: /home/cedriczheng/.cache/pants/setup/bootstrap-Linux-x86_64/1.18.0/bin/python: No such file or directory
➜  c2dw git:(mypants-upgrade) ✗
h

hundreds-father-404

03/17/2022, 7:22 AM
But when I run ./pants test , I got this.
I'm heading to bed, but I'm confused. Are you getting the test issue on Pants 1.17, which you were already using?
o I modified the version to 1.18.0 in pants.ini and run ./pants --version again and then got this.
Did you update the
./pants
script to https://raw.githubusercontent.com/pantsbuild/setup/d1da382f6de0420940ec6007a39cba87c21075c6/pants?
f

freezing-area-97131

03/17/2022, 7:28 AM
Oh damn, I was switching between branch and it turns out I didn’t refresh the
./pants
script. Sorry for bothering! Now the
./pants --version
works fine. Thx! Good night!
That syntax error when running
./pants test
happened in 1.17.0 and 1.18.0 as well.
Oh I figure out the root cause of that! There is this
salesforce-bulk
package depends on
simple-salesforce
depends on
Authlib
, and
Authlib
just got a new version on March 15, 2022, which does not support python2.7 . So I think I need to specify the version of Authlib as well in the
BUILD
. I have already done this before. Now how do I specify the relationship between
simple-salesforce
and
Authlib
? I think the
simple-salesforce
version should be
0.75.3
and
Authlib
should be
0.15.5
.
Copy code
python_requirement_library(
    name='salesforce-bulk',
    dependencies=[
        ':simple-salesforce',
    ],
    requirements=[
        python_requirement(name='salesforce-bulk', requirement='salesforce-bulk==2.0.0'),
    ],
)
https://docs.authlib.org/en/latest/changelog.html#version-1-0
I added this and it seems to solve the above error. Don’t know if I have done right (?
Copy code
python_requirement_library(
    name='simple-salesforce',
    dependencies=[
        ':Authlib',
    ],
    requirements=[
        python_requirement(name='simple-salesforce', requirement='simple-salesforce==0.75.3'),
    ],
)
And then I upgrade all the way to
1.23.0
, I removed the option
[pytest]
in
pants.ini
(cuz the instructions told me to) and then I encountered the same
coverage
version problem again.
Copy code
**** Failed to install coverage-6.3.2 (caused by: NonZeroExit("received exit code 1 during execution of `['/usr/bin/python2.7', '-s', '-', 'bdist_wheel', '--dist-dir=/tmp/tmpmv_55ce0']` while trying to execute `['/usr/bin/python2.7', '-s', '-', 'bdist_wheel', '--dist-dir=/tmp/tmpmv_55ce0']`",)
):
stdout:

stderr:
Traceback (most recent call last):
  File "<stdin>", line 14, in <module>
  File "<string>", line 80
    classifier_list.append(f"Development Status :: {devstat}")
                                                            ^
SyntaxError: invalid syntax


11:28:46 [DEBUG] pants.process.lock:pid=7879: releasing lock: <pants.process.lock.OwnerPrintingInterProcessFileLock object at 0x7d542c503e48>

               Waiting for background workers to finish.
04:28:46 00:04   [complete]
               FAILURE
@hundreds-father-404 Any suggestion~?