https://pantsbuild.org/ logo
Join Slack
Powered by
Hey guys, I have the following question: In `pants...
# general
b
Hey guys, I have the following question: In 
pants
v1 there was a dedicated approach for managing jvm dependencies -> https://v1.pantsbuild.org/3rdparty_jvm.html In 
pants
v2 a new concept of dependencies inference is introduced. (https://blog.pantsbuild.org/automatically-unlocking-concurrent-builds-and-fine-grained-caching-on-the-jvm-with-dependency-inference/) What I'm trying to understand is how pant v2 suggests to handle transitive dependencies & conflicts? Should transitive dependencies. be listed explicitly?
✅ 1
w
transitive dependencies should not need to be listed explicitly as dependencies in 
jvm_artifact
targets: they are available at compile time.
BUT: if you have an 
import
statement for a thirdparty dependency, then it is not actually transitive: for inference to work, everything with an 
import
statement likely needs to actually have a declared 
jvm_artifact
b
import
statements can help resolve 
direct dependencies,
right? (scan imports and find corresponding jvm_artifacts) but what about runtime dependencies -> they can't be inferred cause they are absent in 
import
statements.... For example to compile very simple Hello world application that uses cats-effect its enough to only have 
cats-effect
But when running that application we will face 
ClassNotFound
exception cause there needs to be also 
cats-core
in classpath.
w
is 
cats-core
not a dependency of 
cats-effect
…?
at runtime, transitive dependencies should definitely all be present
b
is 
cats-core
not a dependency of 
cats-effect
…?
cats-effect
depends on 
cats-core
For example if I run same simple app via 
sbt
and only list 
cats-effect
as application dependency - 
cats-core
will also be loaded transitively and added to classpath (which leads to another problem of possible conflicts)
With pants v2 seems like only direct dependency will be added and any transitive depedency that is required at runtime needs to be explicitly added (manually added) ?
w
that should not be the case, no
is 
cats-core
a 
provided
dep of 
cats-effect
perhaps? which version of 
cats-effect
are you using, and which pants goal are you seeing this with?
b
I'll create a very simple project to double check that.
w
(although likely version dependent)
b
I'm using 
cats-effect_2.13: 3.3.14
w
let me remember the coursier invocation for this… one sec.
b
So the story is like this: 1. 
cats-effect
depends on 
cats-effect-kernel
(compile dependency) 2. 
cats-effect-kernel
depends on 
cats-core
(compile dependency)
w
hm yea: should be there:
Copy code
$ cs resolve org.typelevel:cats-effect_2.13:3.3.14
org.scala-lang:scala-library:2.13.7:default
org.typelevel:cats-core_2.13:2.7.0:default
org.typelevel:cats-effect-kernel_2.13:3.3.14:default
org.typelevel:cats-effect-std_2.13:3.3.14:default
org.typelevel:cats-effect_2.13:3.3.14:default
org.typelevel:cats-kernel_2.13:2.7.0:default
org.typelevel:simulacrum-scalafix-annotations_2.13:0.5.4:default
b
so If I understood correctly - 
cats-core
shouldn't be defined in dependencies when running app that already has 
cats-effect
in dependencies however that doesn't seem to work like that currently in pants v2
I can get a simplest working project to double check that
w
@brave-planet-28195: yea, that would be helpful. which goal are you running?
b
run
w
and which version of pants?
b
./pants run src/scala...../service:bin
2.13.0
w
yea, an example would be really helpful. if you 
package
instead, and then unzip the resulting jar, do you see cats-core’s classfiles in there?
./pants package src/scala...../service:bin
b
the current project where I'm switching from sbt to pants is kinda messy I'll create fresh small project and double check
w
thank you: and yea, feel free to file a bug if you reproduce this. it’s definitely unexpected.
b
ok, I created a very simple project https://github.com/ekravchenko/dependency_example with only on dependency (except 
scala-lang
) -> 
cats-effect
And everything works fine - all required dependencies are there and I don't have to list 
cats-core
or anything explicitly...
however
if I add 
cats-core
to 
3rdparty/jvm/org/typelevel/BUILD
with version 
2.8.0
- I can't run 
Main
anymore.
Copy code
jvm_artifact(
    name="cats-core",
    group="org.typelevel",
    artifact="cats-core_2.13",
    version="2.8.0",
    packages=["cats.**"],
)
Copy code
Exception in thread "main" java.lang.NoClassDefFoundError: cats/kernel/Semigroup
w
ah, yea: that probably makes sense. 
3.3.14
must not be compatible with 
2.8.0
b
The cause of the problem is version of cats-core. It is 
2.8.0
The thing is that 
cats-effect:3.3.14
depends on 
cats-core
2.7.0 So that creates a conflict
If I change version of cats-core to 2.7.0 and still dont list it in dependecies - everything will work fine
w
yea. sounds like that is resolved then: huzzah
b
that kinda comes back to my initial question - how dependency conflicts like this are resolved in pants v2. (Cause when importing other libraries like 
fs2
and rest... 
cats-core:2.8.0
will appear on the radar again)
w
v2 uses 
coursier
it looks like coursier highlights the incompatibility, but doesn’t fail by default:
b
yes, for that in 
sbt
for example there is a 
StrictMode
or something like that... in that case build will fail
w
we don’t currently support passing that through to 
coursier
, but it would be fairly easy to add…
(just need to figure out which coursier CLI option enables it)
b
Copy code
The CLI now accepts options to enable and configure the new strict conflict manager.

--strict enables the strict conflict manager,
w
mmm… tricky.
so: we did actually use strict mode by default for a little while: https://github.com/pantsbuild/pants/pull/13353 … but i believe that it was reverted due to a bug. working on finding a link for that.
f
and the upstream coursier issue https://github.com/coursier/coursier/issues/1364 has never been fixed
b
it would be great to have an option to see 
coursier
output of 
resolve
for dependencies of specific target
like 
./pants resolve src/scala..../service:bin
and coursier resolution tree is presented
4 Views
Open in Slack
PreviousNext
Powered by