My repository has a bunch of separate plotly dash applications. Each application can have an assets folder with JavaScript and CSS. When running the application, any files in assets are automatically served up by the application. I'd like to share some common CSS with all applications. So I'd have an assets folder in the root of the repository with some CSS. Wondering if there is a way that I could combine the root assets with the assets in the application when calling pants run. --/repo |-- assets |-- common.css |--myapp |--assets |-- application.css So assets/common.css would become part of myapp/assets at runtime.

I think relocated_files (https://www.pantsbuild.org/docs/reference-relocated_files) might be what you need.

That sounds like what I need but I must be doing something wrong. I have the following in my root BUILD file:

*files*(

name = "common_assets",

sources = [

"assets/**",

],

)

In my application folder (/dash_campaign_status_report/BUILD) I have:

*resources*(

name = "assets",

sources = [

"assets/**",

],

)

*relocated_files*(

name="common",

files_targets=['//:common_assets'],

src='',

dest='dash_campaign_status_report',

)

*pex_binary*(

name = "bin",

dependencies = [

":assets",

":common",

],

entry_point = "<http://dash_campaign_status_report.app|dash_campaign_status_report.app>",

)

When I package :bin I see the assets from the application folder but not the common ones.

👀

A bit more information. When I run ./pants dependencies dash_campaign_status_report:bin, I get these entries related to assets:

dash_campaign_status_report/assets/dash-campaign-status-report.css:../assets

dash_campaign_status_report/assets/dash.css:../assets

dash_campaign_status_report:common_assets

Hm I think this could possibly be weirdness with using files targets with a pex_binary? That's a hunch, not certain Could you please try using the

archive

type? https://www.pantsbuild.org/docs/resources#archive-create-a-zip-or-tar-file You'd keep the

pex_binary

still, and only be adding the new

archive

target. You don't necessarily need to use

archive

at the end of the day, now the goal is solely to verify those files are showing up where you expect in the archive. It will allow us to narrow down the problem

I tried that with the following archive target:

*archive*(

name = "archive",

files = [

":assets",

":common_assets",

],

format="tar",

)

Hmm, could you possibly send a zip file with the setup you're describing? You can strip all content - we only care here about directory structure, along with BUILD file content

Sure. FYI - when I changed the assets target from resource to files then I got all the CSS in the archive. I then tried to package the pex_binary and I got no CSS files

K I think I see some things go on: -- When you have this archive

archive(
    name = "archive",
    files = [
        ":assets",
        ":common_assets",
    ],
    format="tar",
)

You can get all the CSS files to show up correctly in the same folder iff

:assets

is

files

, and

:common_assets

is

relocated_files

. This is expected. Note that the

files

field expects file targets, not resources targets. So the resources targets get ignored. That's pretty subtle - we should add a warning -- For the

pex_binary

, you're right that I can't get the

relocated_files

to show up. I can get

:assets

if it's a

resources

target, but not if it's a

files

target. So there's something going on there. It looks like this is actually intentional. And now I'm remembering that it has to do with source roots. In a PEX, we strip source roots, so

src/python/myorg/app.py

becomes

myorg/app.py

in the final binary. But source roots are never stripped with a `files`/`relocated_files()` target, and it would be incorrect to put

src/assets/data.json

with that prefix iiuc Instead, this is why the

archive

target type exists. You'll create an archive with a PEX file in it, and then the loose CSS files also accompanying the PEX. Then, when you run your PEX, it can access those loose files It will look something like this:

relocated_files(
    name="common_assets",
    files_targets=['assets:common'],
    src='',
    dest='dash_campaign_status_report',
)

files(
    name = "assets",
    sources = [
        "assets/**",
    ],
)

archive(
    name = "archive",
    files = [
        ":assets",
        ":common_assets",
    ],
    packages=[":bin"],
    format="tar",
)


pex_binary(
    name = "bin",
    entry_point = "main.py",
)

Does that make sense?

Can I run that with ./pants run?

Hm, no you cannot. Tricky.. Semi related, what does

./pants roots

say?

I have:

.

plugins

You can package arbitrary files into a

pex

no? Just by having them as

resource

dependencies

Thanks for the help tonight. Looks like I'll probably have to think of another way to make this work. Your last suggestion doesn't quite give me what I need. This is the repository layout:

.

|── BUILD

|── assets

|   └── common.css

└── dash_campaign_status_report

├── BUILD

└── assets

  

├── dash-campaign-status-report.css

  

└── dash.css

If I add a resource target to the root BUILD like this:

resources(

name = "common_assets",

sources = [

"assets/common.css",

],

)

And then I add the resources to the pex_binary target in dash_campaign_status_report/BUILD like this:

pex_binary(

name = "bin",

dependencies = [

...

"//:common_assets",

...

],

entry_point = ...,

)

I get a pex layout that looks more or less the same as the repository layout:

.

|── assets

|   └── common.css

└── dash_campaign_status_report

└── assets

  

├── dash-campaign-status-report.css

  

└── dash.css

What I want is:

.

└── dash_campaign_status_report

└── assets

├── common.css

  

├── dash-campaign-status-report.css

  

└── dash.css

Hmm, I wonder if we should change

pex_binary

to work with files targets @happy-kitchen-89482? I'm actually not certain why it would be problematic to have an unstripped file like

src/assets/img.png

in the PEX? I think it wouldn't work to load via

pgkutil.get_data()

, but maybe

open()

would still work? If you don't recall, I can play around with this

I think we should do that

Okay, I'll play with it later today to see where the limitations are. @high-egg-2153 are you able to workaround this in the meantime? Or put this particular thread on hold till the next dev release?

it would be on the coder to know how to open the file, you coulg use

pkgutil

if you happened to know that the file fell into a package, and

open()

would only work if you were running unzipped

Yeah that's what I'm thinking. We could maybe encourage

resources

because it Just Works more easily, but agreed. And better than the confusing alternative of adding

relocated_resources

I'm good for now. I created some symbolic links to make it work. This will get messy as things grow though so I'll stay tuned.

Didn't get to this today, setting a reminder to look tomorrow

I'm not opening the files. If a plotly dash application has an assets folder, it gets served by the application automatically - https://dash.plotly.com/dash-enterprise/static-assets

ok… then the relevant question is probably whether Plotly Dash supports “zipapps”, where the files are embedded as resources and must be loaded via `pkg_resources`… it’s not clear from that page

It looks like yeah, it does use resources. Greg's motivation for relocated_resources is that he has a common.css he only wants to define once, but to include in multiple distinct packages That sounds sensible to me. It's advanced for sure, but reasonable Certainly this new target type could be tricky for users to get right because of reasoning about source roots, but I suspect they'd know that and would rather have the possibility

And it needs to be at a different relative path in each app? If not, can just define it once using resources without relocating

It needs to be merged with other CSS files. He has the structure <project>/assets. All the CSS needs to go in there. He wants to add in common.css into that pre-existing folder

That should be fine if the source root is correct:

src/resources/assets

His source root is the build root. The final path should be <project>/assets/common.css Right now, it's something like ./common_assets/common.css

I'm current using 2.3 and I will give your PR a try. I can't do it right now but I will try a little later. Thanks!

traditionally this has been accomplished by defining more source roots, similar to defining more PYTHONPATH entries, which all end up merged.

Even though the files are being loaded as resources by dash (IIUC), can't they be packaged using relocated files? Or are the existing relocation primitives not expressive enough to do this?

@enough-analyst-54434 That was Greg's original attempt, which worked great with ./pants run. But it doesn't work with ./pants package because we don't include files in pex_binary. Hence my PR to "fix" that But I'm not comfortable landing my files() PR, given how ./pants run and ./pants package would diverge when source roots are used. It would fix Greg's case because his source root is the repo root, but it wouldn't work for others Hence, me thinking relocated_resources is the way to go

and to be clear: the final location on the pythonpath is supposed to be

${project}/assets/common.css

, rather than

assets/common.css

…?

Yes, that is correct. Where he has >3 distinct projects

so he’d have to relocate the one file differently for each consumer, i guess… not actually possible to do with source roots.

Yeah, one relocated_resources target per project. Which is totally sensible, we expected that to be the case when designing relocated_files

run

and

package

diverging seems like a separate (related) problem, perhaps.

There, I think the proper "fix" would be no longer copying loose files into the chroot for run, but that doesn't seem ideal. The issue is because we strip source roots when packaging a binary, but I think we need to keep doing that iiuc Instead, I plan to address that discrepancy through a warning when you package a pex_binary with files in the closure that those files won't be included, and to use either archive or resources instead

Sorry no. Following the discussion after you had asked I wasn't sure if you still wanted me too. I will give it a try now.

If you could, that'd be great. Thanks! Imo, we should land

relocated_resources

- Dash is loading these things as resources, not as files. And the use case you have is an advanced one easy to mess up, but it's a legit use case