I have run into another issue:

09:31:56 00:13       [create-monolithic-jar]
09:31:56 00:13         [add-internal-classes]
09:31:56 00:13         [add-dependency-jars]
                       ==== stdout ====

                       ==== stderr ====
                       java.lang.IllegalAccessError: class org.pantsbuild.tools.jar.JarBuilder$7 tried to access method 'com.google.common.collect.TreeTraverser com.google.common.io.Files.fileTreeTraverser()' (org.pantsbuild.tools.jar.JarBuilder$7 and com.google.common.io.Files are in unnamed module of loader 'app')
                       	at org.pantsbuild.tools.jar.JarBuilder$7.execute(JarBuilder.java:743)
                       	at org.pantsbuild.tools.jar.JarBuilder.getAdditions(JarBuilder.java:1188)
                       	at org.pantsbuild.tools.jar.JarBuilder.getEntries(JarBuilder.java:1093)
                       	at org.pantsbuild.tools.jar.JarBuilder.write(JarBuilder.java:1009)
                       	at org.pantsbuild.tools.jar.Main.doRun(Main.java:401)
                       	at org.pantsbuild.tools.jar.Main.run(Main.java:367)
                       	at org.pantsbuild.tools.jar.Main.main(Main.java:481)
                       	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                       	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                       	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                       	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                       	at com.martiansoftware.nailgun.NGSession.run(NGSession.java:280)

FAILURE: jar-tool failed

This happens for all cases when I try to bundle/jar a JVM/Java target. I have recently upgraded from 1.17 to 1.26 (migrating py2 to py3). Hopefully there is something simple that I need to change.

Hm. I’m not quite sure how the JVM backend works. @fancy-queen-20734 or @witty-crayon-22786 does this look familiar?

That smells like old Pants picked an old JDK pre-modules support and new Pants picks a newer JDK that supports modules via: "...and com.google.common.io.Files are in unnamed module of loader 'app'".

I believe I have update the

pants

script, but that may have been for an earlier version. We are using java 11 in the platform settings. I can download the latest python script

OK - so you were previously using JDK11 fine pre-upgrade?

Yes, that is correct. We are using the following settings in pants.ini:

[jvm-platform]
default_platform: java11

platforms: {
    'java11': {'source': '11', 'target': '11', 'args': [] },
  }

So that stacktrace is suspicious because the missing guava is not shadowed and we have a general facility for shadowing jvm tools....

We were running v1.17 due to python2 dependencies

s/shadowing/shading/

We are using guava libraries in our

3rdparty/jvm

dependencies if that helps (version

28.2-jre

)

Thanks.

I can definitely try that 🙂

OK - a few minutes to prepare some coherent instructions.

That is correct, the version I am using is 1.26.0

Here's the diff:

diff --git a/src/python/pants/backend/jvm/subsystems/jar_tool.py b/src/python/pants/backend/jvm/subsystems/jar_tool.py
index bd02ef3c5..c6ad8d12d 100644
--- a/src/python/pants/backend/jvm/subsystems/jar_tool.py
+++ b/src/python/pants/backend/jvm/subsystems/jar_tool.py
@@ -10,12 +10,15 @@ from pants.subsystem.subsystem import Subsystem
 class JarTool(JvmToolMixin, Subsystem):
     options_scope = "jar-tool"
 
+    _MAIN = "org.pantsbuild.tools.jar.Main"
+
     @classmethod
     def register_options(cls, register):
         super().register_options(register)
         cls.register_jvm_tool(
             register,
             "jar-tool",
+            main=cls._MAIN,
             classpath=[JarDependency(org="org.pantsbuild", name="jar-tool", rev="0.0.16"),],
         )
 
@@ -24,7 +27,7 @@ class JarTool(JvmToolMixin, Subsystem):
             self.tool_classpath_from_products(
                 context.products, "jar-tool", scope=self.options_scope
             ),
-            "org.pantsbuild.tools.jar.Main",
+            cls._MAIN,
             jvm_options=self.get_options().jvm_options,
             args=args,
             workunit_name="jar-tool",

applied and attempting to bundle a jar

Ah - sorry - you beat me to instructions. Where did you apply this? Should be:

$ PATCH_TARGET=$(find ~/.cache/pants/setup/*/1.26.0_*/lib/*/site-packages/pants/backend/jvm/subsystems/jar_tool.py)

Which for me is:

$ echo $PATCH_TARGET
/home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/1.26.0_py36/lib/python3.6/site-packages/pants/backend/jvm/subsystems/jar_tool.py

Ah, I did a find and found a different one. I patched

/Users/jay/.cache/pants/setup/bootstrap-Darwin-x86_64/1.26.0_py37/lib/python3.7/site-packages/pants/backend/jvm/subsystems/jar_tool.py

and the command you just me gave has

/Users/jay/.cache/pants/setup/bootstrap-Darwin-x86_64/1.26.0_py37/lib/python3.7/site-packages/pants/backend/jvm/subsystems/jar_tool.py

I don't see a difference in those two paths.

It was a symlink. Sorry, was a copy/past error. I was originally patching

/bootstrap-Darwin-x86_64/pants.ymwVoV/install/lib/python3.7/site-packages/pants/backend/jvm/subsystems/jar_tool.py

which is just a symlink

At any rate - let me know when you get results ... if there is still a stacktrace it should have mangled names prefixed with the package

__pants_shaded__

IIRC.

got it. Running now

Ah, its

__shaded_by_pants__

... this is a walk down memory lane.

👍 It seems that the jvm target is hanging on [cache]:

10:32:16 00:03       [create-monolithic-jar]
10:32:16 00:03         [add-internal-classes]
10:32:16 00:03         [add-dependency-jars]
10:32:16 00:03         [cache]
                     No cached artifacts for 1 target.
                     Invalidated 1 target.
10:32:16 00:03         [cache].
                     Using cached artifacts for 1 target.

OK - I did all this cowboy style. Slowing down to apply the patch myself and try building a binary jar...

No worries 🙂 Thanks for helping! Let me know if there is anything I can provide in the meantime (I have some meetings in the afternoon in case I don't immediately respond)

If that doesn’t work, you could try adding some

--add-opens

to jvm options for the jar tool. That might not be very scalable.

will look at that option

OK - my patch introduces a deadlock in the jvm stack that I need to dig into further. You can try to niggle along with Nora's suggestion in the meantime, but I'll dig a bit more a bit later today. I'll probably file an issue to track this shading and its current failure.

Thanks @enough-analyst-54434!

I have not yet had time to circle back to the deadlock issue when shading. I have not forgotten about you though.

Thanks John! No rush 😄

Just dropping in to note I did not get time to circle back here. I likely won't have time until next week unfortunately. I'm hoping @fancy-queen-20734's suggestion to use --add-opens as needed can unblock you in the meantime.

Thanks for keeping me updated. I will try the

--add-opens

sugggestion soon. This is sort of a migration task to a new version, so we can still work using the older pants versions until we can figure out this issue. One note, we use guava as a jvm depency, and have it include in most of our java targets, however, I did notice we have one target (at least one), that was able to bundle as it did not depend on the guava 3rd party dep.