Hi everyone ! After a team pants upgrade version from

1.11.0rc2

to

1.14.0

, I can't compile anything and I get the following error.

16:52:27 00:02     [conan-prep]**** Failed to install setuptools-40.4.3 (caused by: NonZeroExit("received exit code 1 during execution of `['/usr/bin/python3.6', '-', 'bdist_egg', '--dist-dir=/tmp/tmpKDItdj']` while trying to execute `['/usr/bin/python3.6', '-', 'bdist_egg', '--dist-dir=/tmp/tmpKDItdj']`",)
):
stdout:

stderr:
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 5, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils.core'

What is weird is that i can import

distutils.core

from python 3.6 but what I find even weirder is that it is not using pants virtual env defined in

.cache/...

. What I tried: - remove

home/.cache/pants

- remove

.cache

in working directory -

./pants clean-all

Does that ring any bell ?

Huh. Could you try using 1.13.0 please and seeing if that works as a debugging step?

The compilation works with

1.13.0

Okay great. If you wouldn’t mind, it would be really helpful if you could please try bisecting which version this breaks in from the 1.14.0.x series by trying out the different versions from https://www.pantsbuild.org/notes-1.14.x.html, e.g.

1.14.0.dev0

I’m not quite sure what is going on and will help if we can narrow down the release it broke in.

I ll do that so I start from 1.14.0.dev0 up to the stable release ?

Yes please, that would be great! To save you time, you can try to “bisect” it by doing a manual binary search - e.g. start at 1.14.0.dev3, then you know whether the issue is before 1.14.0.dev3 or after it and you can ignore half of the releases. Let’s say the issue does not happen in 1.14.0.dev3, then try 1.14.0.dev1

Haha that was i was going for 😃

@aloof-angle-91616 do you think https://github.com/pantsbuild/pants/pull/7060 is at play? It looks like the failure is happening at

conan-prep

i don’t think so because that change doesn’t affect the conan prep task at all just the actual conan-fetch task which consumes it? not sure if i’m missing anything

@flaky-photographer-75744 what led you to this conclusion:

but what I find even weirder is that it is not using pants virtual env defined in `.cache/...`.

['/usr/bin/python3.6', '-', 'bdist_egg', '--dist-dir=/tmp/tmpKDItdj']

while we use

python2.7

Hm, the

./pants

script being out of date shouldn’t cause any issues with 1.14.x. Although I do recommend updating it by running

curl -L -O <https://pantsbuild.github.io/setup/pants>

, regardless of if you’re upgrading right away to 1.15.0. We made several improvements and it means you’ll be able to upgrade to 1.15.0+ without issue.

python tools will use the default interpreter constraints for the repo to resolve the package, or should be. the task might still be to blame, trying to figure it out i just woke up so may be wrong

Are you using

--python-setup-interpreter-constraints

?

It fails with the new

./pants

also

Please try doing that to see if it resolves anything. Try adding to your

pants.ini

[python-setup]
interpreter_constraints: ["CPython==2.7.*"]

👏

that makes sense then

Yay! So the way that works, it says your overall project should use Python 2.7 for any subprocesses like running tests or running your code. You can also set compatibility constraints for specific targets. For example, maybe your overall code could use either Python 2 or Python 3, but this particular target only supports Python 2 no matter what. See https://www.pantsbuild.org/python_readme.html#configure-the-python-version for more info on this all

Alright, it's ok for me but should we consider this a bug ? It's not happening on my colleague computers and what will happen when we drop python2 ?

Yes I think potentially. I’m trying to figure out why this only started happening for you on 1.14.0.dev2 also

Thanks a lot for your awesome answer time !

@flaky-photographer-75744 can you do one more sanity check for me please? Do you have any Python tests in your codebase? If so, can you please temporarily remove

interpreter_constraints

from your

pants.ini

and run

./pants test path/to:target

, then tell us which Python interpreter was being used (should print to the top of the test output)

We don't have any python tests in our code base, only a python file but it is a resource

Hm okay. What I suspect is happening is that you were using Python 3 all along and everything worked fine, until 1.14.0.dev2 made Python 3 stop working, e.g. as the result of bumping some of Pants’ dependency versions. Trying to confirm this is true that you were using Python 3 all along Will think of another way to test this

I have to go, I will continue to look at this tommorow

Okay we figured out what’s going on with why this broke in 1.14.0.dev2, and why it doesn’t break for your coworker. When you leave off

--python-setup-interpreter-constraints

, Pants gives a default value. Until 1.14.0.dev2, we defaulted to Python 2. This meant that even with you leaving off the setting, any subprocess would always use Python 2. But then in 1.14.0.dev2, we changed the default to either 2.7 or 3.6+. So, suddenly Pants started to use Python 3.6. Does your coworker have Python 3.6+ installed on their computer? I suspect not, which would explain why Pants is going with Python 2.7 for them and not running into this issue. It’s still an open question why you’re getting the underlying issue of distutils not being found

So the problem relies in my python3 installation ?

A little unclear why Python 3 is failing to find distutils. We wouldn’t expect that to happen