Hi, have a question on tasks. We're working on som...
# general
d
Hi, have a question on tasks. We're working on some support for publishing python targets, and running into some issues with the wiring. We have the following: `job_target`: this is conceptually similar to a
python_binary
. Thas a single 'main'
python_library
, an entrypoint, and some other params `publish_task`: this is similar to the existing publish, but uses twine. All these steps work well individually. The issue here is getting them to play nicely with each other. What I'd like to do is call our
deploy-job
goal with the
job_target
as a target. This would then in turn call our publish goal, which calls setup.py. The issue that I'm running into is that when setup.py is called, it fails with the error :
FAILURE: setup-py target(s) must provide an artifact.
I imagine this is because it is looking at the
job_target
for a provides argument, which it doesn't have. What we're looking for is to somehow specify that the
round_manager.require_data('python_dists')
line can inform setup.py libraries that this needs to be run against are the python_library deps of
job_target
I've dabbled in the python_binary code to see how this is done, but couldn't exactly grab a hold. Happy to provide more details as well
fwiw a
job_target
looks roughly like this:
Copy code
python_job(
  name="my_job",
  job_name="Job 1",
  entrypoint="myjob.entrypoint',
  dependencies=[
    ":lib",
  ],
)
where
:lib
is a python library, specifically, the python library that we want to run a recursive setup.py command on.
It seems like this might be the magic line https://github.com/pantsbuild/pants/blob/33a6d51db91ea1fe69c39117c8878abcb824cdfb/src/python/pants/backend/python/tasks/setup_py.py#L665 Should our target somehow be specifying that this dependency should be pushed onto the target_roots? It doesn't seem like python_target messes with the roots, so I feel like that's not the move
Alright, just to update here after some debugging, this seems like it may be more difficult than expected. I tried running this against just a simple python_library target, and the target_roots seems to only be specified at the library level. The way the code in setup_py is structured is it takes the specified library, and if it has a setup.py block, then it runs it recursively, I imagine this wouldn't work either if I did the following: 1. create a target with multiple python libraries that all have setup.py
provides
blocks, but doesn't have one itself 2. run setup.py against that target
Which seems 'reasonable' on the outset, but it also means you wouldn't be able to do something like call setup.py on a target aggregate building each package in the aggregate list
@wonderful-iron-54019