https://pantsbuild.org/ logo
Join Slack
Powered by
How would people feel about adding `experimental_s...
# development
b
How would people feel about adding 
experimental_shell_command
metadat to specify the 
type
of asset? I need it to output a 
resource
because I can't package a 
file
šŸ˜­
FYI @hundreds-father-404 @curved-television-6568 Also FYI @bulky-alarm-94027 just was testing our strategy for handling the 
git
stuff (admittedly I wired it recently) and hit this, so TL;DR it still won't work šŸ˜­
h
it always does a 
file
atm?
b
AFAICT
Copy code
class GenerateFilesFromShellCommandRequest(GenerateSourcesRequest):
    input = ShellCommandSourcesField
    output = FileSourceField
šŸ‘ 1
c
I guess itā€™s that, or if resources may depend on file targets to ā€œturnā€ them into resources.. šŸ¤·
b
There's a semantic difference between the two fo things like python asset inference, or (in this case) PEX packaging. So I'm guessing it wont be a "one-size-fits-all"
e
Isn't not being able to package a 
file
a very old bug? There is no reason you should not be able to package a 
file
- makes perfect sense.
b
According to the error message:
Copy code
Filesystem APIs like `open()` are not able to load files within the binary itself; instead, they read from the current working directory.
e
Yeah, but that's just wrong.
Its telling you to be dumber than you are.
I think we guard-railed a common case and with it took away a reasonable case.
šŸ‘ 1
h
I wrote the error message and admit I've never really understand it all 100%. I tried several ways to load a 
file
and couldn't ever get it working I'd be very happy to remove the restriction
e
There is really nothing to be done to get it working - just add the file.
The semantics are up to the user.
Instead of stripping the source root you strip the buildroot. That should be the only difference.
b
@enough-analyst-54434 I really don't follow, can you elaborate? If the file isn't packaged into the PEX, how should I load it when being run?
Unless you're suggesting we DO package it into the PEX. In which case, fiesta
e
It should always have been packaged in the PEX.
It was in v1.
b
I see. Then there's sort of little distinction of files vs resources for Python, no?
e
I think some users who didn't know Python that well / zippapps got confused & so we went overboard with padded walls.
files vs resources is the same distinction in all langs I know. There is a resource API the lang provides to access data via the lang namespace system; and then there are files to access with file APIs.
Same in Pythin & Java at any rate.
b
So it's really just a matter of convention. I could load a file via 
pkgutil
and a resource via 
open
if I know the right values. I think maybe then there is no useful distinction from Pants' perspective
āž• 1
(We've invented conventions in 
asset inference
and PEX packaging, but that's arbitrary)
e
PEX does not have the distinction fwiw. When you add files the layout is your choice.
b
Fair. So the only distinction then is just the guardrail itself. And the guardrail protects you from testing your code which uses 
open
and succeeds because the test sandbox uses loose files, but then the code fails in deployment because your layout is the zipapp one. Did I get that right?
(and back on topic, nit that 
experimental_shell_command
doesn't tell you what it does... it generates files)
w
And the guardrail protects you from testing your code which uses 
open
and succeeds because the test sandbox uses loose files, but then the code fails in deployment because your layout is the zipapp one.
Did I get that right?
yep
the guardrail is necessary, but a way to re-classify 
files
to 
resources
would be helpful. part of what it would need to do is maybe controlling which namespace/package the file went into when it became a resource.
b
the guardrail is necessary
Is it though? For 
test
the loose files exist either way. There's no discernible difference in the 
test
sandbox. So a user can use either, and they can also use 
open
and 
pkgutil
. Then it comes time to package. It still doesn't really matter unless my code is being packaged by PEX and deployed as a zipapp. Which also disregarding other PEX layouts, or even non-PEX deployments. OTOH unifying into one 
asset
type eliminates a lot of confusion and code IMO.
We could still warn for one 
asset
target if: ā€¢ the asset is not under a source root ā€¢ you are packaging a PEX and are using a zipapp layout
Instead of making the user choose confusingly between 
file
v 
resource
we choose the behavior for them (we're much more likely to get it right šŸ˜‰) and then we know exactly when to warn because we dont have to trust the user used the correct target type
w
For 
test
the loose files exist either way. Thereā€™s no discernible difference in the 
test
sandbox. So a user can use either, and they can also use 
open
and 
pkgutil
.
resources have source roots applied: i.e., they live in packages, and have a prefix strippedā€¦ files do not, and will be at their absolute path in the repo
b
(Ah right I have no source roots, I thought I did lol)
Open in Slack
PreviousNext
Powered by