https://pantsbuild.org/ logo
Join Slack
Powered by
Pants_build_fails_with_DistUtilsError_Cython_.m
# general
t
Pants_build_fails_with_DistUtilsError_Cython_.m
Pants_build_fails_with_DistUtilsError_Cython_.m
Appreciate any help/ guidance. Thanks.
e
Two questions for you: 1. What Pants version are you using? 
./pants --version
2. Does your python TLS test pass on all pythons on your PATH? 
which -a python{,2,3}
? I'm guessing the answer to 2 is no and Pants is picking one of the old TLS pythons to run with, but hopefully you can help confirm.
t
Hi @enough-analyst-54434 Thanks for your response. For question #1, pants version is "1.15.0". For #2, I get the following paths in my terminal: /usr/local/bin/python /usr/bin/python /usr/local/bin/python /usr/local/bin/python2 /usr/local/bin/python2
e
Ok. If you can use each of those pythons with your TLS test, that would be great. I predict one or more fails.
t
Hi @enough-analyst-54434 your prediction is absolutely right. When I did TLS test on each of those python paths, everything passed but one. $/usr/local/bin/python -c "import json, urllib2; print json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']" TLS 1.2 $/usr/bin/python -c "import json, urllib2; print json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']" TLS 1.0 $/usr/local/bin/python -c "import json, urllib2; print json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']" TLS 1.2 $/usr/local/bin/python2 -c "import json, urllib2; print json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']" TLS 1.2 Is there any configuration that I should do in my pants.ini to instruct pants to use per se “/usr/local/bin/python” path instead of “/usr/bin/python”? Appreciate your help. Thanks.
e
Ok, /usr/bin/python is likely Apple's and you have a fairly old OSX at this point. There are a few approaches. One is to restrict the search path, but that only works if this is a personal repo or else all developers / machines can be made to conform. Another is to restrict the python version, but similar constraints apply. For completeness, what are the python versions here? IE ask each of those interpreters for its 
--version
.
t
Hi @enough-analyst-54434 Thanks again, I checked on the version. Most of the interpreters return me the latest version 2.7.16, except one. $/usr/local/bin/python --version Python 2.7.16 $/usr/bin/python --version Python 2.7.10 $/usr/local/bin/python --version Python 2.7.16 $/usr/local/bin/python2 --version Python 2.7.16 I’m not behind an enterprise firewall or enterprise repository. I’m using my home/ personal internet connection — if that helps.
e
It does. You can take your pick of one of these edits to pants.ini: 1. Constrain the python version to avoid known bad:
Copy code
[python-setup]
# Avoid picking the Apple provided 2.7.10 interpreter that lacks new enough TLS support to interact with pypi.
interpreter_constraints: ["CPython>2.7.10"]
2. Constrain the search path to known good:
Copy code
[python-setup]
# Ensure we just look at brew managed pythons which have new enough TLS support to interact with pypi.
interpreter_search_paths: ["/usr/local/bin"]
Of the two options I'd personally use the second. Its probably a safer bet that Brew managed pythons will be in better shape than Apple's given historical precedence.
t
Thanks @enough-analyst-54434 It will give it a shot in order of constraint #2 first, and if need be constraint #1 next. I’m also thinking about upgrading my OS X from Sierra to Mojave and see how that pans out. I’ll keep posted about the outcome.
Hi @enough-analyst-54434 - I’d like to really thank you a lot for your suggestions, it worked. For me, actually the constraint #1 “interpreter_constraints: ['CPython>2.7.10']” did the trick. In spite of making the above edits to pants.ini - in fact it kept throwing the very old TLS error. I reckoned perhaps the changes/ edits were not taking into effect, so I did “./pants clean-all” and rerun it, and it finally worked. Meaning, build succeeded and I now see “dist” folder and I have my target pex file in it. Just one last question though, appreciate your inputs. Should I necessarily create 2 more BUILD files - 1 each in “spouts” and “bolts” folder below? Following is my complete source code folder structure: $ls -l drwxr-xr-x 4 Arunsmacbook wheel 136 Jun 4 22:18 3rdparty drwxr-xr-x 3 root wheel 102 Jun 6 22:23 dist -rwxr-xr-x 1 Arunsmacbook wheel 6894 May 5 20:34 pants -rw-r--r-- 1 Arunsmacbook wheel 242 Jun 6 22:16 pants.ini drwxr-xr-x 4 Arunsmacbook wheel 136 Apr 14 17:42 src $ls -l src/main/python/ -rw-r--r-- 1 Arunsmacbook wheel 289 Jun 4 22:16 BUILD drwxr-xr-x 5 Arunsmacbook wheel 170 Apr 26 10:21 bolts -rw-r--r-- 1 Arunsmacbook wheel 883 Apr 26 19:16 pmTop.py drwxr-xr-x 4 Arunsmacbook wheel 136 Apr 26 18:05 spouts $ls -l src/main/python/spouts/ -rw-r--r-- 1 Arunsmacbook wheel 48 Apr 26 18:06 init.py -rw-r--r-- 1 Arunsmacbook wheel 703 Apr 16 23:00 ecg.py $ls -l src/main/python/bolts/ -rw-r--r-- 1 Arunsmacbook wheel 110 Apr 26 10:26 init.py -rw-r--r-- 1 Arunsmacbook wheel 504 Apr 16 23:53 getRate.py -rw-r--r-- 1 Arunsmacbook wheel 891 Apr 26 19:16 putRate.py $cat src/main/python/spouts/__init__.py all = ['ecg'] from .ecg import EcgReadings I did adhere to 3rdparty pattern prescribed in Pants as shown below: $ls -l 3rdparty/python/ total 16 -rw-r--r-- 1 Arunsmacbook wheel 22 May 24 00:02 BUILD -rw-r--r-- 1 Arunsmacbook wheel 53 Jun 4 22:19 requirements.txt However, I did not create separate BUILD files with “python_library” target for my “spouts” and “bolts” folder. Per my understanding, since I have “__init__.py” files in those folders - which in turn performs the required “imports” (as shown above), the pex file created by pants should work seamlessly. Does that make sense? Please advise. Thanks.
e
Aha. I do believe the 
clean-all
you had to perform has since been fixed for 1.16+. BUILD files, or rather the targets you define inside them, can be thought of as primarily demarcating logical units. For BUILD file targets for code, those units tend to map onto packages (or the equivalent structure in whatever underlying language the code is written in). A full software project generally composes many packages that depend on each other in various ways. If you model the software project to Pants with on BUILD file / 1 target, then Pants can only act on the whole project at once. If the project contains 10 production files and 5 test files for example, changing one production file and running 
./pants --changed-parent=master test
will run all tests in all 5 test files. This is unfortunate, since it may be the case that only 1 test file depends on the 1 production file that was changed. Since you only let Pants know about the 1 large target, it can (currently) know no better. If, instead, you generally stick to 1 target per package (see 111 advice in the 'Target Granularity' section here: https://www.pantsbuild.org/build_files.html), then Pants can generally do less work. All that said - in a small python project, 1 BUILD file is probably fine.
4 Views
Open in Slack
PreviousNext
Powered by