pantsbuild #general

Channels

welcome

github-notifications

announce

limited-jordan-75746

08/30/2018, 3:46 PM

Copy code

jar_library(name='poi-jars',
jars=[
    jar(org='org.apache.poi', name='poi', rev='3.13'),
    jar(org='org.apache.poi', name='poi-contrib', rev='3.7-beta3'),
    jar(org='org.apache.poi', name='poi-ooxml', rev='3.13'),
  ],
)

jar_library(name='tika-jars',
jars=[
    jar(org='org.apache.tika', name='tika-core', rev='1.3'),
  ],
)

java_library(name='tika',
    dependencies=[
        ':tika-jars',
    ],
    sources = "placeholders",
    excludes = [
        exclude('edu.ucar', 'netcdf'),
        exclude('org.apache.poi','poi'),
        exclude('org.apache.poi','poi-scratchpad'),
        exclude('org.apache.poi','poi-ooxml'),
    ]
)

And then the consuming library like:

Copy code

scala_library(
  name = “my-project”,
  sources =
       globs("src/main/scala/**/*.scala"),
  dependencies = [
          '3rdparty/jvm:tika',
          '3rdparty/jvm:poi-jars',
          '
       ]
)

limited-jordan-75746

08/30/2018, 3:48 PM

When I run

./pants compile my-project

I get

cannot find symbol

for everything in POI. It looks like that pattern seems to exclude POI completely from

my-project

instead of just from the

tiki

library.

limited-jordan-75746

08/30/2018, 3:48 PM

Is this by design? If so, is there an alternative aside from using the intransitive argument and hand picking everything (I’m worried about how we’ll be able to manage that given the number of conflicts we have)

curved-vase-73456

08/30/2018, 9:33 PM

regarding the recent topic of transitive dependencies and this passage from docs:

More subtly, if you have many intermediate
java_library
targets between your
jvm_binaries
and your
jar_library
targets (which is normaly the case), simply changing which combination of
java_library
targets are in the same
./pants
invocation may invalidate the cache and force Pants to recompile them, even if their sources are unchanged. This is because they may resolve different versions of their transitive jar dependencies than the previous time they were compiled, which means their classpaths will be different. Getting a different classpath causes a cache-miss, forcing a recompile. In general recompiling when the classpath changes is the correct thing to do, however this means that unstable transitive dependencies will cause a lot of cache-thrashing. If you have a large repository with a large amount of code, recompiles get expensive.

How one can detect which dependencies/3rdparty jars are causing this behavior? I see a message when invoking strict_deps “check for conflicting dependencies with `-Ylog-classpath`” but the resulting output is quite huge and I’m not sure how to see/search for conflicts using it 😕

witty-crayon-22786

08/31/2018, 12:26 AM

@curved-vase-73456: that compiler message is not a helpful one, unfortunately. all it is indicating is that the compiler needs more than it has been provided.

witty-crayon-22786

08/31/2018, 12:27 AM

so you need to add additional dependencies for that target.

curved-vase-73456

08/31/2018, 12:42 AM

@witty-crayon-22786 ty for the response! As I understand instead of following

strict_deps

approach it’s possible to configure

managed_jar_libraries

and/or

managed_jar_dependencies

to achieve the same result (avoiding recompilation on changing combination of targets). So should it be done for every

3rdparty

jar or is there a way to tell which dependency can produce possible conflicts?

witty-crayon-22786

08/31/2018, 12:48 AM

We don't use

managed_jar_libraries

, so I don't know a lot about it... sorry.

curved-vase-73456

08/31/2018, 12:48 AM

ah, ok. what do you use?

strict_deps

witty-crayon-22786

08/31/2018, 12:56 AM

For all of the Java code, and some of the Scala code, yea.

aloof-angle-91616

08/31/2018, 2:18 AM

imho,

strict_deps

makes a jvm build significantly easier to understand and maintain, implicit transitive deps are hard for me to follow personally

aloof-angle-91616

08/31/2018, 2:19 AM

also, i released

1.9.0rc2

, which should be the final release candidate for

1.9.0

. i'm sorry for the delay, this was my first time doing such a release (just because people asked about it itc last week)

aloof-angle-91616

09/03/2018, 9:33 PM

if anyone uses emacs and pants, i'm asking about what to do with an ensime gen plugin and other emacs stuff here https://github.com/pantsbuild/pants/issues/6445

user

09/04/2018, 6:41 PM

Error build kotlin project in android studio I am trying to learn Kotlin. i updated Android studio to the latest version an di created a sample project. but when i tried to edit the layout file,and build the project, I received the below posted error messages. please let me know how to fix these errors error messages: Gradle project sync failed. basic functionalities (e.g editing, debugging) will not work properly. Design editor is unavailable until a successful build gradle file // Top-level build file where you can add...

enough-analyst-54434

09/04/2018, 7:12 PM

CI is currently broken by a

coverage

unstable version publish yesterday (unfortunately, although we pin our direct dep on

pytest-cov

, it in-turn lets its

coverage

dep float with

coverage>=3.7.1

). I'll have a fix up shortly.

magnificent-coat-60998

09/04/2018, 8:06 PM

Hello. Is there a way to run a shell command, e.g. a code editor, inside a prepared python environment similar to the

go-env

task?

aloof-angle-91616

09/05/2018, 10:13 AM

ci was broken again if you had rebased master in the wrong few hours today, i hadn’t rebased master for two days against a PR that i merged today and two specific test failures crept in, which should be fixed by https://github.com/pantsbuild/pants/commit/bff18f5d9761cada2521dd0e30ce74cf53e84e6d

fierce-park-88503

09/05/2018, 1:36 PM

is there a way to list all targets of a particular type (e.g.,

python_tests

? or just any test targets?)

average-vr-56795

09/05/2018, 1:42 PM

./pants list

takes some flags which may be helpful

average-vr-56795

09/05/2018, 1:42 PM

Off the top of my head, you want something like

./pants list --filter=python_tests ::

average-vr-56795

09/05/2018, 1:42 PM

That’s totally a lie

average-vr-56795

09/05/2018, 1:44 PM

./pants filter --filter-type=python_tests ::

💯 1

fierce-park-88503

09/05/2018, 6:15 PM

oooh TIL about filter, nice

wide-energy-11069

09/05/2018, 6:16 PM

Hi @enough-analyst-54434 heads up https://github.com/pantsbuild/pants/commit/ab5aeeb34512f50fdaf4e01bf005559c3ccbe08d is breaking, I am still working on the repro. The symptom is for

SQLalchemy

user with the error:

Copy code

sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'part_factory.vendor_id' could not find table 'vendor' with which to generate a foreign key to target column 'id'

wide-energy-11069

09/05/2018, 6:17 PM

the error msg comes in too late probably, so still need to dig down what the real issue is

enough-analyst-54434

09/05/2018, 6:18 PM

OK, yeah that seems super far removed from that change. If you come up with a repro I'm happy to help.

enough-analyst-54434

09/05/2018, 6:19 PM

The most likely thing seems to me to be adding that dep on the requirements.txt file invalidated a resolve

wide-energy-11069

09/05/2018, 6:19 PM

also relevant part is that

./pants run

works, but building the pex then run the pex does not.

enough-analyst-54434

09/05/2018, 6:19 PM

And the real problem is a different version of sqlalchemy than prior?

wide-energy-11069

09/05/2018, 6:19 PM

no, verified that they are the same version