https://pantsbuild.org/ logo
Join Slack
Powered by
In my open source project (using Python and Scala)...
# general
b
In my open source project (using Python and Scala) using Pants: https://github.com/liga-ai/liga I want to migrate the Scala part from sbt to pants, but last time I tried 
deploy_jar
https://github.com/pantsbuild/pants/issues/17790 , I failed to exclude Apache Spark. Am I doing something wrong or it is not supported in Pants？What is the 
deploy_jar
equiv of 
sbt package
and 
sbt assembly
?
Recently, there are three pull requests on the JVM backend: • https://github.com/pantsbuild/pants/pull/17251 • https://github.com/pantsbuild/pants/pull/17663 • https://github.com/pantsbuild/pants/pull/17419 cc @witty-family-13337 @ancient-vegetable-10556 Need help from you!
https://github.com/liga-ai/liga-vision is also using Scala and Python
h
@ancient-vegetable-10556 is on vacation. @witty-family-13337 may be around though!
Re https://github.com/pantsbuild/pants/issues/17790, it looks like your BUILD filet is trying to shade Apache Spark, not exclude it?
Are you trying to remove it from the deploy jar?
w
Excluding Spark is not yet possible. We need to support scopes or something similar (not fully aware on any progress on that). This is because Spark is still needed to compile the project, we just don’t want it to be part of the final JAR.
h
And we have no way of making it "provided" or anything like that. Yeah.
w
shading it with the zap rule looks like a workaround attempt to achieve the same, but the final jar will still contain other transitive dependencies from Spark
h
So shading it is the wrong thing to do, it sounds like
because the client code will use the shaded version, not the provided version, and so any runtime classname magic will fail
w
well, the zap rule will remove any classes under the given pattern
h
yes, but won't it rewrite the callers to use the shaded version?
w
that’s a good point, and in case of zap, there isn’t a shaded version as it was removed
b
Are you trying to remove it from the deploy jar?
Yes.
Well, 
sbt package
and 
sbt assembly
seems to be the blocking functionality of Pants JVM (Scala) support
h
Or am I misunderstanding how zap works
w
well, those sbt goals are supported with 
pants package …
applied to a 
deploy_jar
. Just that the functionality is incomplete due to lack dependency scopes
b
any plan to solve the dep scopes problem? is it hard?
h
I don't think it's crazy hard, just not done yet
The implementation wouldn't be hard, I think it's more around design. How we want to represent the concept of dependency scopes in pants.
E.g., what would a BUILD file look like that specifies scopes...
b
Looking forward to the non-experimental Scala support!
5 Views
Open in Slack
PreviousNext
Powered by