hello! I've inherited a codebase that uses Pants ...
# plugins
f
hello! I've inherited a codebase that uses Pants 1.26.0, Python 3.6.12, and Cython 0.28.2 (yeah, I know...) I'm trying to chip away at getting all of this upgraded, but my first priority is getting to Python 3.7 because 3.6 is EOL and no longer receiving security updates the Cython part of the build happens using a plugin that a former coworker wrote, apparently with help from someone in the Pants community. this was originally in May 2018 and the mailing list archives don't seem to stretch back that far. if I just upgrade to 3.7 (changing interpreter_constraints in pants.toml, and running Pants inside a Py3.7-based Docker container instead of the 3.6-based one we use now), with no other changes, the build fails with:
Copy code
Traceback (most recent call last):
  File "/srv/submodules/obscura-plugins/plugins/setup.py", line 4, in <module>
    from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'

Exception message: Command '['/usr/bin/python3.7', '/srv/submodules/obscura-plugins/plugins/setup.py', 'build_ext', '--inplace', '--verbose']' returned non-zero exit status 1.
is there a "simple" fix to get this working under 3.7? if I have to upgrade Pants and/or Cython to get it to work I will, but my goal in the short term is to do the minimum necessary to get upgraded to 3.7, and then tackle the rest of the tech debt separately pants.toml: http://ix.io/3PIe register.py of the plugin: http://ix.io/3PIi compile_cython.py: http://ix.io/3PIj BUILD containing the Cython target: http://ix.io/3PIk the setup.py living next to that BUILD file, where the exception above is being thrown from: http://ix.io/3PIm
e
@fast-notebook-4005 does
/srv/submodules/obscura-plugins/plugins/setup.py
represent your code or third party code?
If its your code, does it specify Cython in
setup_requires
? If its not your code can you offer up a PyPI link?
f
that is my code, it's the last of those ix.io links...it doesn't seem to have
setup_requires
anywhere
though it's failing from the import at the top of the file, so sticking
setup_requires
somewhere below that doesn't seem like it would help
e
Well, there are two paths: 1. You use setup_requires and lazify everything you need. If there is an import, hide it in function calls. 2. Use pyproject.toml / PEP-517 / PEP-518
For 2 I need to check if that old Pants' old Pex can handle those ...
For 1 - I'm not sure how this ever worked for you unless Pants / Pex was buggy and let a system Python 3.6 interpreter pre-installed copy of Cython to leak in. Pants and Pex definitely don't leak like this currently.
Ok, Pants 1.26.0 is on Pex 2.1.7 which does support
pyproject.toml
. So try adding a
pyproject.toml
to that distribution (you can keep
setup.py
, that will still get run) with this contents:
Copy code
[build-system]
requires = ["setuptools>=40.8.0", "wheel", "Cython==0.28.2"]
build-backend = "setuptools.build_meta:__legacy__"