https://pantsbuild.org/ logo
#general
Title
# general
s

straight-waiter-86090

03/16/2017, 8:03 PM
hi - is it possible to use unpack-jars to unpack zip files? I do the following and the ivy retrieval works fine, but no unpacking is attempted:
Copy code
jar_library(name='tomcat', jars=[ jar('org.apache.tomcat', 'tomcat', '8.0.41-r9', intransitive=True, ext='zip') ])

unpacked_jars(libraries=[':tomcat'])
e

enough-analyst-54434

03/16/2017, 9:00 PM
dyoung: do relevant targets in the pants command line depend on the un-named (so default-name of the parent dir) unpacked_jars target?
basically targets in BUILD files are not imperative verbs, they just form a graph and if the verbs you list on the pants command line can't walk to that node in the graph, it will have no effect.
s

straight-waiter-86090

03/16/2017, 9:02 PM
I was just writing up an example that doesn’t depend on my zip:
Copy code
jar_library(name='groovy', jars=[ jar('org.codehaus.groovy', 'groovy-binary', '2.2.2', intransitive=True, ext='zip') ])
unpacked_jars(name="unpack_groovy", libraries=[’:groovy’])
Copy code
./pants -ldebug unpack-jars :unpack_groovy
so I’m trying to run the unpack-jars task on the unpacked_jars target.
e

enough-analyst-54434

03/16/2017, 9:04 PM
So, I'm recalling this target now, introduced and used by Square, but did you read here?: http://www.pantsbuild.org/build_dictionary.html - search for unpacked_jars - 2 hits
s

straight-waiter-86090

03/16/2017, 9:06 PM
I see the two hits
e

enough-analyst-54434

03/16/2017, 9:06 PM
Yup
So, I'm sure unpack-jars is working, but its unpacking to an inscrutable location under .pants.d/
Its intended for use with remote_sources which IIRC in the Square case was used for .thrift files fed to protoc via the gen goal
s

straight-waiter-86090

03/16/2017, 9:08 PM
If I do the same thing with a .jar I get debug output. I also tried searching .pants.d and don’t see anything for the zip case.
Copy code
jar_library(name='lang', jars=[ jar('commons-lang', 'commons-lang', '2.6', intransitive=True) ])
unpacked_jars(name="unpack_lang", libraries=[':lang'])
Then I get this output:
Copy code
Unpacking jar commons-lang:commons-lang:2.6::jar from /Users/dyoung/proj/pants/.pants.d/ivy/jars/commons-lang/commons-lang/jars/commons-lang-2.6.jar to /Users/dyoung/proj/pants/.pants.d/unpack-jars/unpack-jars/.unpack_lang.
e

enough-analyst-54434

03/16/2017, 9:09 PM
But so, what do you want to do with the unpacked thing
compile its contents? embed in a larger jar as resources, or something else?
As it stands, the design is remote_sources produces a synthetic target of your choice type - say java_library, with the unpacked files as its sources
s

straight-waiter-86090

03/16/2017, 9:10 PM
I have a custom task that creates symlinks in a runtime. right now just to source files, but I would like to link to unzipped files and/or generated files (generated by other custom tasks).
e

enough-analyst-54434

03/16/2017, 9:11 PM
ok - I think then you need to make your task unpacked_jars aware
s

straight-waiter-86090

03/16/2017, 9:11 PM
specifically I’d like to unzip the tomcat zip and copy/symlink the files from it into my runtime directory.
e

enough-analyst-54434

03/16/2017, 9:11 PM
let me give you a pointer to some code that does this today
s

straight-waiter-86090

03/16/2017, 9:12 PM
I’m happy to try it. I just can’t find evidence that anything is getting unzipped in the zip case.
and there is no debug output.
You may need to dig in and find out why the zip extensioned jar_library is not making it into JarImportProducts
s

straight-waiter-86090

03/16/2017, 9:16 PM
OK. I was going to try to run pants from source, add some debugging in this area. If I can find a bug I can enter an issue on it.
e

enough-analyst-54434

03/16/2017, 9:16 PM
I need to run, but once you have that bit figured out - zip extension issue - the relevant code for a Task depending on and using an unzipped archive is here: https://github.com/pantsbuild/pants/blob/42893403743a399c89fa7371d8f5ec8ed2ab836d/src/python/pants/core_tasks/deferred_sources_mapper.py#L48-L60
Thanks for digging in Derek
s

straight-waiter-86090

03/16/2017, 9:16 PM
thanks for your help!
Hi. I think I got it sorted out. It was the ivy resolution code, not the unpacking code. https://github.com/pantsbuild/pants/pull/4339/commits/155b790caf717d6cad57355b4d47c361add65fc2
5 Views