Hi folks, I tested my simple Presto UDF project wi...
# general
b
Hi folks, I tested my simple Presto UDF project with Maven: https://github.com/jiegzhan/prestoUdfs/tree/master/src/main, then I uploaded "prestoUdfs-1.0-SNAPSHOT.jar" to my EMR Presto cluster, it worked. When I ran "jar tf prestoUdfs-1.0-SNAPSHOT.jar", it looks like this:
Copy code
META-INF/
META-INF/MANIFEST.MF
META-INF/services/
com/
com/roku/
com/roku/dea/
com/roku/dea/presto/
com/roku/dea/presto/udfs/
META-INF/services/com.facebook.presto.spi.Plugin
com/roku/dea/presto/udfs/UdfPlugin.class
com/roku/dea/presto/udfs/ProductUDF.class
META-INF/maven/
META-INF/maven/com.roku.dea/
META-INF/maven/com.roku.dea/prestoUdfs/
META-INF/maven/com.roku.dea/prestoUdfs/pom.xml
META-INF/maven/com.roku.dea/prestoUdfs/pom.properties
Since our team is using pants to build the project, so I copied my files to Pants project folder, and have a BUILD file like this:
Copy code
resources(name='prestoUdfs-lib-resources',
  sources=['src/main/resources/META-INF/services/com.facebook.presto.spi.Plugin']
)

java_library(
    name='prestoUdfs-lib',
    dependencies=[provided('3rdparty/java/com/facebook:presto-spi'),
                  provided('3rdparty/java/com/google:guava')
                 ],
    sources=globs('*.java'),
)

jvm_binary(
    name='prestoUdfs-artifacts',
    dependencies=[':prestoUdfs-lib',
                  ':prestoUdfs-lib-resources'],
)
Then I built the project, it says
Copy code
WARN] Globs did not match. Excludes were: []. Unmatched globs were: ["src/main/java/com/roku/dea/presto/udfs/src/main/resources/META-INF/services/com.facebook.presto.spi.Plugin"].
Then I moved META-INF folder to the same folder as .java files, and changed BUILD file to
Copy code
resources(name='prestoUdfs-lib-resources',
  sources=['META-INF/services/com.facebook.presto.spi.Plugin']
)

java_library(
    name='prestoUdfs-lib',
    dependencies=[provided('3rdparty/java/com/facebook:presto-spi'),
                  provided('3rdparty/java/com/google:guava')
                 ],
    sources=globs('*.java'),
)

jvm_binary(
    name='prestoUdfs-artifacts',
    dependencies=[':prestoUdfs-lib',
                  ':prestoUdfs-lib-resources'],
)
Now my jar file looks like
Copy code
META-INF/
META-INF/MANIFEST.MF
com/
com/roku/
com/roku/dea/
com/roku/dea/presto/
com/roku/dea/presto/udfs/
com/roku/dea/presto/udfs/META-INF/
com/roku/dea/presto/udfs/META-INF/services/
com/roku/dea/presto/udfs/META-INF/services/com.facebook.presto.spi.Plugin
com/roku/dea/presto/udfs/UdfPlugin.class
compile_classpath/
compile_classpath/src.main.java.com.roku.dea.presto.udfs.prestoUdfs-lib.txt
com/roku/dea/presto/udfs/ProductUDF.class
Looks like META-INF/services/com.facebook.presto.spi.Plugin is not on the root like Maven jar, so my EMR presto cluster can't load this jar. Is there a way to put resources to root path? Thanks for your help.
m
I think `bundle`s are what you should be using instead of
resource
the
relative_to
clause along with the
fileset
should be able to dictate where the file should be located w/ respect to the jar
b
I tried bundle, but META-INF path still looks like this
Copy code
META-INF/
META-INF/MANIFEST.MF
com/roku/dea/presto/udfs/META-INF/
com/roku/dea/presto/udfs/META-INF/services/
com/roku/dea/presto/udfs/META-INF/services/com.facebook.presto.spi.Plugin