pantsbuild #general

hi - I previously asked about why the remote cache doesn’t seem to be working for me. I did compare options and couldn’t find any relevant differences (things like python path were different). I do find a different value of

sources.files_hash

for source field in

target.payload

. When I compute the sha1s of the files and filenames they match, but yet files_hash doesn’t. Still pursuing why it would be.

witty-crayon-22786

05/19/2017, 6:07 PM

thanks for the heads up. you might also see whether you reproduce this with

--no-enable-v2-engine

witty-crayon-22786

05/19/2017, 6:07 PM

if so, would want to make it a

1.3.0

blocker

straight-waiter-86090

05/19/2017, 6:21 PM

I’ll try that now.

straight-waiter-86090

05/19/2017, 7:00 PM

The cache works for me again with

enable_v2_engine: False

straight-waiter-86090

05/19/2017, 7:00 PM

I can enter an issue with as much info as I have.

straight-waiter-86090

05/19/2017, 7:00 PM

I wrote a small task to dump out some pieces of the target that show the difference.

straight-waiter-86090

05/19/2017, 7:25 PM

https://github.com/pantsbuild/pants/issues/4610

gifted-beard-62952

05/21/2017, 7:27 PM

Hey all... i wrote my own task for pants to generate python wheels out of a certain package. The issue is that i am seeing the following error when i execute the task:

gifted-beard-62952

05/21/2017, 7:28 PM

@User uploaded a file: Untitled

-.txt

gifted-beard-62952

05/21/2017, 7:29 PM

And i was able to trace the problem all the way back to: https://github.com/pantsbuild/pants/blob/master/src/python/pants/cache/local_artifact_cache.py#L143

gifted-beard-62952

05/21/2017, 7:29 PM

What i don't understand is that the task does

maybe_create_results_dir

and then this removes the created dir, which makes

ensure_legal

to fail

gifted-beard-62952

05/21/2017, 7:30 PM

and that happens at the time when my custom task calls

with invalidated(....

gifted-beard-62952

05/21/2017, 7:30 PM

Is this an issue? or can someone provide any insights on this because it seems to me that at some points pants will fail because of itself.

witty-crayon-22786

05/22/2017, 4:39 PM

@straight-waiter-86090 : re #4610: are the relevant targets using

globs

witty-crayon-22786

05/22/2017, 4:45 PM

@gifted-beard-62952 : you should not need to clear or create the results_dir if

cache_target_dirs

is on

witty-crayon-22786

05/22/2017, 4:46 PM

if you're seeing otherwise, please file an issue, with info on which of `cache_target_dirs`/`create_target_dirs` you have enabled

straight-waiter-86090

05/22/2017, 4:50 PM

yes, the targets use

globs

. Most use

globs('*.java')

. In the case I included in the bug it used

globs

with a small list of files as arguments.

witty-crayon-22786

05/22/2017, 4:50 PM

so something like

globs('LiteralFile.java', 'Another.java')

witty-crayon-22786

05/22/2017, 4:50 PM

witty-crayon-22786

05/22/2017, 4:51 PM

(we have a testcase for ordering of literal files, so i'm just trying to narrow in precisely here)

straight-waiter-86090

05/22/2017, 4:51 PM

The case from the bug did this:

straight-waiter-86090

05/22/2017, 4:51 PM

Copy code

message_support_code=[
    'com/r9/nucleus/message/MessageBundleMapBuilder.java',
    'com/r9/nucleus/message/MessageHelperWrapper.java',
    'com/r9/nucleus/message/MessageBundle.java',
    'com/r9/nucleus/message/IMessageHelper.java',
    'com/r9/nucleus/message/LocaleContext.java',
    'com/r9/nucleus/message/MessageDebug.java',
    'com/r9/nucleus/commons/Context.java',
    'com/r9/nucleus/commons/ContextKeys.java',
    'com/r9/nucleus/commons/ContextKey.java',
    'com/r9/nucleus/commons/Singleton.java',
    'com/r9/nucleus/commons/ResettableSingleton.java',
    'com/r9/nucleus/commons/DisposableObject.java',
    'com/r9/nucleus/commons/DisposableRegistry.java',
]

java_library(
    name='nucleus_message_support',
    dependencies=[
        'alien/3rdparty:guava',
        scoped('alien/3rdparty:ehcache', scope='compile'),
        'alien/3rdparty:spring-core' ],
  sources=globs(*message_support_code),
  strict_deps=True,
  platform='java8'
)

witty-crayon-22786

05/22/2017, 4:51 PM

gotcha.

straight-waiter-86090

05/22/2017, 4:52 PM

Most of our targets are like this, though:

straight-waiter-86090

05/22/2017, 4:52 PM

Copy code

java_library(
    dependencies=[
        scoped('nucleus/java:nucleus_message_support', scope='compile'),
        scoped('alien/3rdparty:spring-web', scope='compile'),
        scoped('nucleus/java/com/r9/nucleus/logging', scope='compile'),
        scoped('nucleus/java:nucleus-2', scope='compile'),
        scoped('nucleus/java:nucleus-3', scope='compile'),
        scoped('nucleus/java:nucleus-4', scope='compile'),
        scoped('nucleus/java:nucleus-4-msgs', scope='compile'),
        scoped('alien/3rdparty:servlet-api', scope='compile'),
        'nucleus/java/com/r9/nucleus/autovalidate',
    ],

    sources=globs('*.java'),
    strict_deps=True,
    platform='java8'
)