(this isn't necessarily a pex question, but it is about python packaging in pants -- let me know if i should move to #general) i'm trying to adding an option that evaluates

setup.py

files for pants

python_dist()

targets in a context that gives them access to pants code (in this case, so that the setup.py can import a pants module which extends distutils/setuptools classes. it is entirely possible to write the distutils/setuptools extensions as a completely separate pypi module, as they don't currently need to rely on other parts of pants, and could then just be accessed using

setup_requires

in the

python_dist()

target (and this is actually pretty reasonable). however, it seems like it should be possible to somehow expose the running pants environment to the setup.py invocation we do in

BuildLocalPythonDistributions

and/or

SetupPyRunner

, so the setup.py can just do:

from setuptools import find_packages
from distutils.core import Extension
from pants.some.module import setup

setup(
  # ...,
  ext_modules=[Extension(...)],
  packages=find_packages(),

I assume this would be done either by creating a pex with the pants requirement, or by adding something to

PYTHONPATH

in the executing environment for the setup.py. this probably isn't that hard when using pants in any other repo besides the pants repo because we can expect a pex with all the relevant pants code to be available, since that is how pants is released, but i can't figure out how to expose the python code from the running pants to a setup.py invocation (i may be trying to go about this all wrong). there's a very little more context in https://github.com/pantsbuild/pants/issues/5661

Right now setup.py is run directly with a python interpreter with minimal sys.path modification via Installers. You'd want to just switch to building a loose pex of the python_dist and executing setup.py using that pex

i just realized we could just make the

python_dist()

depend on a

pants_requirement()

in the

setup_requires

kwarg in the BUILD file which already works great thanks to chris -- my concern was how to make it reflect the running version of pants without having to write down the version string anywhere, mostly

You need nothing so indirect. Simply depending on pants targets is enough.

well,

python_dist()

can't depend on

python_library()

yet

Aha, ok. That should probably be fixed.

currently it's just c/c++ targets actually

The SetupPy task has the logic needed. That could be extracted.

i would do it but i don't know the model of what would be expected to happen (would we require the

python_library()

to have a

provides=

? would we copy the sources over into the dist before building?)

Yeah, see the model in SetupPy. The graph is walked and and dependencies that do not provides are sucked in, if they do provides, they're added as install_requires.

this is all very exciting

Logic is bundled up here: https://github.com/pantsbuild/pants/blob/master/src/python/pants/backend/python/tasks/setup_py.py#L122

great!