https://pantsbuild.org/ logo
Powered by
#general
  • g

    great-midnight-1044

    07/18/2017, 7:37 PM
    right... I see. The value is in ensuring that noone tampered with the artifact.
  • w

    witty-crayon-22786

    07/18/2017, 7:37 PM
    regarding https://github.com/sbt/zinc/pull/371 : you might be able to make an argument for a 32 bit hash if you're certain it's not going to be used in a distributed environment.
  • w

    witty-crayon-22786

    07/18/2017, 7:37 PM
    you would not want it as a distributed cache key, so attaching that caveat nearby would be good.
  • g

    great-midnight-1044

    07/18/2017, 7:38 PM
    why is a 32bit hash not fit for a distributed environment?
  • w

    witty-crayon-22786

    07/18/2017, 7:39 PM
  • w

    witty-crayon-22786

    07/18/2017, 7:39 PM
    @great-midnight-1044 : in the context of one zinc process operating on ~10k analysis files at a time, a collision is pretty unlikely
  • w

    witty-crayon-22786

    07/18/2017, 7:40 PM
    in the context of a distributed storage system holding millions of analysis files, 4 billion possible hashes just isn't enough.
  • w

    witty-crayon-22786

    07/18/2017, 7:43 PM

    http://preshing.com/images/probability-distribution.png▾

  • g

    great-midnight-1044

    07/18/2017, 7:56 PM
    The hash is only run for jars, do you think this scenario is still feasible?
  • g

    great-midnight-1044

    07/18/2017, 7:57 PM
    I see your concern now. I'll think about how we can work around it...
  • g

    great-midnight-1044

    07/18/2017, 7:58 PM
    The only problem is that file hash expects only one int. Maybe I can tweak it so that it accepts a long and it's bincompat.
  • w

    witty-crayon-22786

    07/18/2017, 7:58 PM
    imo, make it binary.
  • w

    witty-crayon-22786

    07/18/2017, 7:59 PM
    and only stringify it for the text-based format.
  • w

    witty-crayon-22786

    07/18/2017, 7:59 PM
    or lazily, for debugging.
  • g

    great-midnight-1044

    07/18/2017, 8:00 PM
    What do you mean by binary?
  • w

    witty-crayon-22786

    07/18/2017, 8:00 PM
    in protobuf 
    bytes
    , and in memory 
    Array[Byte]
    or protobuf's 
    ByteString
  • g

    great-midnight-1044

    07/18/2017, 8:00 PM
    Are you talking about the analysis representation?
  • w

    witty-crayon-22786

    07/18/2017, 8:00 PM
    (better the latter)
  • w

    witty-crayon-22786

    07/18/2017, 8:00 PM
    yes.
  • g

    great-midnight-1044

    07/18/2017, 8:02 PM
    The issue I was describing is in the API of 
    FileHash
    , that takes 
    int hash
    as a parameter.
  • g

    great-midnight-1044

    07/18/2017, 8:03 PM
    Even before, when we used sha-1, we were truncating by taking the hash code, which is even more inswcure than truncating xxhash.
  • g

    great-midnight-1044

    07/18/2017, 8:03 PM
    So that is: the file hash was effectively the hash code of the hex representation of the file contents... Pretty ugly.
  • w

    witty-crayon-22786

    07/18/2017, 8:04 PM
    gotcha.
  • w

    witty-crayon-22786

    07/18/2017, 8:04 PM
    ...
  • g

    great-midnight-1044

    07/18/2017, 8:04 PM
    If we want to make the hash 64 bytes long, we will need to change that :) I'll see how I can do it when I get home.
  • w

    witty-crayon-22786

    07/18/2017, 8:04 PM
    yea, wow.
  • w

    witty-crayon-22786

    07/18/2017, 8:04 PM
    @great-midnight-1044 : we can move this to zinc-contrib
  • g

    great-midnight-1044

    07/18/2017, 8:05 PM
    Yeah, better.
  • g

    great-midnight-1044

    07/18/2017, 8:05 PM
    Will be there in a bit, need to eat something.
  • b

    brash-cricket-7489

    07/19/2017, 5:48 PM
    @fast-nail-55400 do you know when we can land https://github.com/toddgardner/pants-plugins? I would like to contribute some improvements on the docker targets
1...178017811782...2420Latest