https://pantsbuild.org/ logo
Powered by
b

brave-planet-28195

10/05/2022, 8:44 PM
Hey guys, I have the following question: In 
pants
v1 there was a dedicated approach for managing jvm dependencies -> https://v1.pantsbuild.org/3rdparty_jvm.html In 
pants
v2 a new concept of dependencies inference is introduced. (https://blog.pantsbuild.org/automatically-unlocking-concurrent-builds-and-fine-grained-caching-on-the-jvm-with-dependency-inference/) What I'm trying to understand is how pant v2 suggests to handle transitive dependencies & conflicts? Should transitive dependencies. be listed explicitly?
✅ 1
w

witty-crayon-22786

10/05/2022, 9:32 PM
transitive dependencies should not need to be listed explicitly as dependencies in 
jvm_artifact
targets: they are available at compile time.
BUT: if you have an 
import
statement for a thirdparty dependency, then it is not actually transitive: for inference to work, everything with an 
import
statement likely needs to actually have a declared 
jvm_artifact
b

brave-planet-28195

10/05/2022, 10:03 PM
import
statements can help resolve 
direct dependencies,
right? (scan imports and find corresponding jvm_artifacts) but what about runtime dependencies -> they can't be inferred cause they are absent in 
import
statements.... For example to compile very simple Hello world application that uses cats-effect its enough to only have 
cats-effect
But when running that application we will face 
ClassNotFound
exception cause there needs to be also 
cats-core
in classpath.
w

witty-crayon-22786

10/05/2022, 10:04 PM
is 
cats-core
not a dependency of 
cats-effect
…?
at runtime, transitive dependencies should definitely all be present
b

brave-planet-28195

10/05/2022, 10:10 PM
is 
cats-core
not a dependency of 
cats-effect
…?
cats-effect
depends on 
cats-core
For example if I run same simple app via 
sbt
and only list 
cats-effect
as application dependency - 
cats-core
will also be loaded transitively and added to classpath (which leads to another problem of possible conflicts)
With pants v2 seems like only direct dependency will be added and any transitive depedency that is required at runtime needs to be explicitly added (manually added) ?
w

witty-crayon-22786

10/05/2022, 10:13 PM
that should not be the case, no
is 
cats-core
a 
provided
dep of 
cats-effect
perhaps? which version of 
cats-effect
are you using, and which pants goal are you seeing this with?
b

brave-planet-28195

10/05/2022, 10:15 PM
I'll create a very simple project to double check that.
w

witty-crayon-22786

10/05/2022, 10:15 PM
(although likely version dependent)
b

brave-planet-28195

10/05/2022, 10:15 PM
I'm using 
cats-effect_2.13: 3.3.14
w

witty-crayon-22786

10/05/2022, 10:16 PM
let me remember the coursier invocation for this… one sec.
b

brave-planet-28195

10/05/2022, 10:18 PM
So the story is like this: 1. 
cats-effect
depends on 
cats-effect-kernel
(compile dependency) 2. 
cats-effect-kernel
depends on 
cats-core
(compile dependency)
w

witty-crayon-22786

10/05/2022, 10:18 PM
hm yea: should be there:
Copy code
$ cs resolve org.typelevel:cats-effect_2.13:3.3.14
org.scala-lang:scala-library:2.13.7:default
org.typelevel:cats-core_2.13:2.7.0:default
org.typelevel:cats-effect-kernel_2.13:3.3.14:default
org.typelevel:cats-effect-std_2.13:3.3.14:default
org.typelevel:cats-effect_2.13:3.3.14:default
org.typelevel:cats-kernel_2.13:2.7.0:default
org.typelevel:simulacrum-scalafix-annotations_2.13:0.5.4:default
b

brave-planet-28195

10/05/2022, 10:21 PM
so If I understood correctly - 
cats-core
shouldn't be defined in dependencies when running app that already has 
cats-effect
in dependencies however that doesn't seem to work like that currently in pants v2
I can get a simplest working project to double check that
w

witty-crayon-22786

10/05/2022, 10:22 PM
@brave-planet-28195: yea, that would be helpful. which goal are you running?
b

brave-planet-28195

10/05/2022, 10:22 PM
run
w

witty-crayon-22786

10/05/2022, 10:22 PM
and which version of pants?
b

brave-planet-28195

10/05/2022, 10:22 PM
./pants run src/scala...../service:bin
2.13.0
w

witty-crayon-22786

10/05/2022, 10:24 PM
yea, an example would be really helpful. if you 
package
instead, and then unzip the resulting jar, do you see cats-core’s classfiles in there?
./pants package src/scala...../service:bin
b

brave-planet-28195

10/05/2022, 10:28 PM
the current project where I'm switching from sbt to pants is kinda messy I'll create fresh small project and double check
w

witty-crayon-22786

10/05/2022, 10:41 PM
thank you: and yea, feel free to file a bug if you reproduce this. it’s definitely unexpected.
b

brave-planet-28195

10/05/2022, 10:50 PM
ok, I created a very simple project https://github.com/ekravchenko/dependency_example with only on dependency (except 
scala-lang
) -> 
cats-effect
And everything works fine - all required dependencies are there and I don't have to list 
cats-core
or anything explicitly...
however
if I add 
cats-core
to 
3rdparty/jvm/org/typelevel/BUILD
with version 
2.8.0
- I can't run 
Main
anymore.
Copy code
jvm_artifact(
    name="cats-core",
    group="org.typelevel",
    artifact="cats-core_2.13",
    version="2.8.0",
    packages=["cats.**"],
)
Copy code
Exception in thread "main" java.lang.NoClassDefFoundError: cats/kernel/Semigroup
w

witty-crayon-22786

10/05/2022, 10:55 PM
ah, yea: that probably makes sense. 
3.3.14
must not be compatible with 
2.8.0
b

brave-planet-28195

10/05/2022, 10:55 PM
The cause of the problem is version of cats-core. It is 
2.8.0
The thing is that 
cats-effect:3.3.14
depends on 
cats-core
2.7.0 So that creates a conflict
If I change version of cats-core to 2.7.0 and still dont list it in dependecies - everything will work fine
w

witty-crayon-22786

10/05/2022, 10:57 PM
yea. sounds like that is resolved then: huzzah
b

brave-planet-28195

10/05/2022, 10:58 PM
that kinda comes back to my initial question - how dependency conflicts like this are resolved in pants v2. (Cause when importing other libraries like 
fs2
and rest... 
cats-core:2.8.0
will appear on the radar again)
w

witty-crayon-22786

10/05/2022, 10:58 PM
v2 uses 
coursier
it looks like coursier highlights the incompatibility, but doesn’t fail by default:
b

brave-planet-28195

10/05/2022, 11:00 PM
yes, for that in 
sbt
for example there is a 
StrictMode
or something like that... in that case build will fail
w

witty-crayon-22786

10/05/2022, 11:01 PM
we don’t currently support passing that through to 
coursier
, but it would be fairly easy to add…
(just need to figure out which coursier CLI option enables it)
b

brave-planet-28195

10/05/2022, 11:05 PM
Copy code
The CLI now accepts options to enable and configure the new strict conflict manager.

--strict enables the strict conflict manager,
w

witty-crayon-22786

10/05/2022, 11:08 PM
mmm… tricky.
so: we did actually use strict mode by default for a little while: https://github.com/pantsbuild/pants/pull/13353 … but i believe that it was reverted due to a bug. working on finding a link for that.
f

fast-nail-55400

10/06/2022, 1:24 AM
and the upstream coursier issue https://github.com/coursier/coursier/issues/1364 has never been fixed
b

brave-planet-28195

10/06/2022, 8:56 AM
it would be great to have an option to see 
coursier
output of 
resolve
for dependencies of specific target
like 
./pants resolve src/scala..../service:bin
and coursier resolution tree is presented
4 Views
Powered by