I've just run into another variant of the <this sa...
# general
r
I've just run into another variant of the this same issue:
Copy code
MODELS = ["aaa", "bbb]

for model_name in MODELS:
    docker_image(
        name=model_name,
        extra_build_args=[
            "BUILD_NUMBER",
            "GIT_COMMIT",
            f"MODEL_NAME={model_name}",
            "IMAGE_VERSION",
        ],
        image_tags=["{build_args.IMAGE_VERSION}"],
        image_labels={
            "GIT_COMMIT": "{build_args.GIT_COMMIT}",
            "IMAGE_VERSION": "{build_args.IMAGE_VERSION}",
        },
        source="Dockerfile",
    )

target(
    name="images",
    description="A target that is used to depend on all model docker images",
    dependencies=[f":{model_name}" for model_name in MODELS],
)
I have this for loop in my build file and a generic target that allows depending on all of the images. (would've used parametrize, didn't want to write the common
extra_env_vars
over and over) If I depend on the individual docker_image targets, then the
docker_info.json
file is materialised in the sandbox. If I depend on the generic target
:images
then I would expect that all of the docker info files would be materialised, but instead, none of them are. How can I tell whether a dependency will / won't materialise files in the sandbox?
h
When you say "If I depend on the individual docker_image targets" - what target type is depending on them, and what cmd are you running? I'm guessing that whatever it is, is only looking at direct deps and not indirect ones (and there may be a good reason for this, would have to look at the code to know)
r
I am running the test goal and referring to these docker images as
runtime_package_dependencies
- I have a test suite that runs against each of the built docker images as a kind of smoke test. What I want to be able to do is declare the list of models in one place, but instead I'm having to list each of the images as a runtime package dependency of the test suite:
Copy code
resource(source="compose.yml", name="docker-compose-config")

python_tests(
    name="tests",
    dependencies=[
        ":docker-compose-config",
        "src/service/edge/test_data:python",
    ],
    runtime_package_dependencies=[
        # I would love to do this:
        "src/service/edge:images",
        # Instead I have to do this:
        "src/service/edge:aaa",
        "src/service/edge:bbb",
    ],
)
h
Thanks, very helpful! This is the problem - it isn't following transitive deps but just looking at direct deps. It probably should not follow transitive deps naively, because packages can depend on other packages (e.g., a docker image on its base image) and we don't want to treat those transitive packages as runtime package deps. However it does make sense to collapse interstitial "target" targets for this purpose.
That should be a straightforward change if you'd like to attempt it? Basically add graph traversal that stops when encountering a target type that isn't
GenericTarget
🫡 1
The change would be entirely within the @rule I link to above
r
I've had a crack at this here as it's my first contribution, feedback is very much appreciated!
❤️ 1
h
Appreciate it! I'll take a look