proud-dentist-22844
05/19/2021, 12:31 AMdependencies
In the old Makefile, in one of the targets that happened before running tests, it looped over contrib/runners/*
and ran python setup.py develop --no-deps
(deps were already installed in another step). How do I do something similar?
ie: Whenever running any of the tests under https://github.com/st2sandbox/st2/tree/pants/st2common/tests,
I need to make sure that each of the packages under https://github.com/st2sandbox/st2/tree/pants/contrib/runners is installed.dependencies=["contrib/runners/*"]
in st2common/tests/BUILD's python_library() declaration, but that resulted in:
DifferingFamiliesError: Expected AddressMaps to share the same parent directory 'contrib/runners/*', but received: 'contrib/runners/action_chain_runner/BUILD'
contrib/runners/*_runner/*_runner
But depending on contrib/runners
complains that there are no BUILD files in that dir (which there aren't).enough-analyst-54434
05/19/2021, 12:42 AMpython setup.py develop ...
) instead of just having the tests depend on the code directly?proud-dentist-22844
05/19/2021, 12:42 AMenough-analyst-54434
05/19/2021, 12:44 AMpython setup.py develop
in a Pants repo.proud-dentist-22844
05/19/2021, 12:44 AMenough-analyst-54434
05/19/2021, 12:45 AMproud-dentist-22844
05/19/2021, 12:45 AMst2common.runners.runner
entrypoint, and then have the tests include all of the directories that have that entrypointYou should be able to remove all setup.py from the repo.I plan to, but not until pants is merged into the main repo.
enough-analyst-54434
05/19/2021, 12:47 AMproud-dentist-22844
05/19/2021, 12:48 AMenough-analyst-54434
05/19/2021, 12:49 AMproud-dentist-22844
05/19/2021, 12:49 AMenough-analyst-54434
05/19/2021, 12:49 AMpython setup.py develop
, but other means might be fine.proud-dentist-22844
05/19/2021, 12:50 AMpython_distribution
inform anything with dep inference? Or some kind of target I can depend on?enough-analyst-54434
05/19/2021, 12:52 AMproud-dentist-22844
05/19/2021, 12:52 AMenough-analyst-54434
05/19/2021, 12:52 AMproud-dentist-22844
05/19/2021, 12:55 AMenough-analyst-54434
05/19/2021, 12:55 AMpython setup,.py develop
all over again, If that step is slow, thats a problem.proud-dentist-22844
05/19/2021, 12:58 AMruntime_package_dependencies
a conftest.py hook?enough-analyst-54434
05/19/2021, 12:59 AMruntime_package_dependencies
just gets you packages built and deposited at $CWD.proud-dentist-22844
05/19/2021, 12:59 AMenough-analyst-54434
05/19/2021, 1:00 AMpytest_sessionstart
is probably appropriate for this: https://docs.pytest.org/en/6.2.x/reference.html#bootstrapping-hooksproud-dentist-22844
05/19/2021, 1:25 AMstevedore(name="st2common.runners.runner", entry_points=["action-chain = action_chain_runner.action_chain_runner"])
target that defines the entrypoints (in each of the runners), and then add python_library(stevedore_dependencies=["st2common.runners.runner"])
which would trigger a rule that adds the python files as dependencies, and somehow injecting the entrypoint metadata.enough-analyst-54434
05/19/2021, 1:36 AMproud-dentist-22844
05/19/2021, 1:37 AMenough-analyst-54434
05/19/2021, 1:40 AMproud-dentist-22844
05/19/2021, 1:41 AMenough-analyst-54434
05/19/2021, 1:46 AMproud-dentist-22844
05/19/2021, 1:47 AMenough-analyst-54434
05/19/2021, 1:49 AMdist-name-dist-version.dist-info/entrypoints.txt
proud-dentist-22844
05/19/2021, 1:50 AMenough-analyst-54434
05/19/2021, 1:52 AMproud-dentist-22844
05/19/2021, 1:54 AMenough-analyst-54434
05/19/2021, 1:55 AMProcess
.proud-dentist-22844
05/19/2021, 1:59 AMenough-analyst-54434
05/19/2021, 2:00 AMproud-dentist-22844
05/19/2021, 4:43 AMstevedore_extension
target that defines the entry_point metadata (using a lot of logic lifted from pex stuff).
Now, I need to generate that entry_points.txt file, but to do that, I need all of the stevedore_extension
targets in a BUILD file, not just the one that triggers the GenerateSourcesRequest
.
Is there a good idiom for generating sources for multiple targets in the same rule? I imagine I could do some kind of Get()
to get all of those targets, but then I'm concerned about each of the GenerateSourcesRequest
saying that they conflict because they generate the same file (though the contents should be the same). Is that the way to go?@rule(desc="Generate entry_points.txt from stevedore_extension target metadata", level=LogLevel.DEBUG)
async def generate_entry_points_txt_from_stevedore_extension(
request: GenerateEntryPointsTxtFromStevedoreExtensionRequest,
) -> GeneratedSources:
# similar to relocate_files, this isn't standard codegen.
# It uses the metadata on targets as source instead of source files.
sibling_targets = await Get(
Targets, AddressSpecs([SiblingAddresses(request.protocol_target.address)]),
)
stevedore_targets = [
tgt for tgt in sibling_targets if tgt.has_field(StevedoreEntryPointsField)
]
resolved_entry_points = await MultiGet(
Get(ResolvedStevedoreEntryPoints, ResolveStevedoreEntryPointsRequest(tgt.entry_points))
for tgt in stevedore_targets
)
entry_points_txt_path = "" # TODO: where does this go?
entry_points_txt_contents = ""
stevedore_extension: StevedoreExtension
for i, stevedore_extension in enumerate(stevedore_targets):
namespace: StevedoreNamespaceField = stevedore_extension[StevedoreNamespaceField]
entry_points: StevedoreEntryPointsField = resolved_entry_points[i].val
if not entry_points:
continue
entry_points_txt_contents += f"[{namespace.value}]\n"
for entry_point in entry_points.value:
# TODO: I need the resolved entry points here...
entry_points_txt_contents += f"{entry_point.name} = {entry_point.value.spec}\n"
entry_points_txt_contents += "\n"
entry_points_txt_contents = entry_points_txt_contents.encode("utf-8")
snapshot = await Get(
Snapshot,
Digest,
CreateDigest([FileContent(entry_points_txt_path, entry_points_txt_contents)]),
)
return GeneratedSources(snapshot)
Process
here because I wasn't working with files, just target metadata... wdyt?
Also, where do I put the dist-name-dist-version.dist-info/entrypoints.txt
so that it gets included in the pexes of sources that get built for pytest? Or does it just need to be alongside the code and by virtue of being on the PYTHONPATH, the dist-info gets picked up?enough-analyst-54434
05/19/2021, 4:17 PMproud-dentist-22844
05/19/2021, 4:18 PM