Q. When there are resource files for different arc...
# general
r
Q. When there are resource files for different archtiectures (e.g., prebuilt binaries for x86-64 and aarch64), is there any way to include them selectively when building packages for a specific platform?
h
Hi, there is not currently via BUILD files, but it possible with plugins. This is how downloading tools like
Protoc
works I think it'd be cool if we supported this via BUILD files. I'm not sure what an ideal design would look like. Any thoughts? -- In the meantime, are you blocked? We can help w/ writing a little plugin
r
If I could run a specific shell script before/after specific build targets, this could be side-stepped in that way, but I think pants won't be happy with arbitrarily removed files, etc.
It would be great if this could be resolved by writing a small plugin
I have already setup-py-generation customization plugin
In my case, the resource files are retrieved as a part of git checkout using LFS, and the checkout'ed tree has binary resources for all architectures.
Prior to using Pants, a shell script has generated
<http://MANIFEST.in|MANIFEST.in>
to selectively include binary resources for a specific target architecture.
h
If I could run a specific shell script before/after specific build targets
Have you seen this page? https://www.pantsbuild.org/docs/run-shell-commands
r
um, the link says 404 not found
h
fixed
👍 1
r
but i'm aware of the experimental shell commands
though, i believe that it would be better to resolve this issue using plugins or whatever more "managed" way
shell scripts should be a kind of last resort 😉
would there be any possibility to hook into the MANIFEST (or something like package-data section) generation steps of the setup-py generator?
h
would there be any possibility to hook into the MANIFEST (or something like package-data section) generation steps of the setup-py generator?
How is that working? Generally, you would likely create a new target type, maybe like this:
Copy code
platform_specific_resource(
   linux_x86_source="linux.ext",
   macos_x86_source="mac.ext",
)
Then, you will use "codegen". It looks similar to https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/docker/util_rules/dockerfile.py, where you create a
GenerateSourcesRequest
. But it would be
input=MyCustomSourceField
and
output=ResourceSourceField
. You'll request the type
Platform
in the rule and figure out which file to load based on that
👀 1
r
or, could i define my own
platform_specific_resources()
target which resembles the vanilla
resources()
target? (where I can control the target platform using a CLI argument or environment variable)
h
Where I can control the target platform using a CLI argument or environment variable
You could do that too if you want. For CLI arguments, you would set up a subsystem and then consume it in your rule https://www.pantsbuild.org/docs/rules-api-subsystems For env variable, https://www.pantsbuild.org/docs/rules-api-process#environment-variables
👀 1
👀 1
r
ok, I'll look into you references
I have question on organizing multiple different platforms (os-arch) pairs. For instance, your target example uses
linux_x86_source
and
macos_x86_source
for a single file. But in this case I need to declare custom fields for all possible combinations of the platforms. How could I avoid that?
Can I add something like field of field?