https://pantsbuild.org/ logo
Powered by
#development
  • a

    average-vr-56795

    10/11/2017, 4:34 PM
    Thanks 🙂
  • w

    witty-crayon-22786

    10/11/2017, 4:39 PM
    i think it would be worth potentially rebooting from the angle @average-vr-56795 mentioned above (as we discussed offline)... where you move the listing/manifest of artifacts->blob into the pants binary, or into BUILD files in the case of overrides, and then fetch them out of a store. i think the only bit that would still need design would be how you would key the platform specific bits: currently 
    name+version+binary_name
    is a key, and then the rest of the mapping is defined by the store... would be more challenging to put the whole mapping in a pants option, or in a BUILD file, etc.
  • w

    witty-crayon-22786

    10/11/2017, 4:40 PM
    but perhaps you could standardize a stored manifest of some kind... maybe a protobuf 
    Directory
    in the current layout.
  • w

    witty-crayon-22786

    10/11/2017, 4:40 PM
    lower priority than isolation / remoting, but definitely a thing.
  • r

    rough-minister-58256

    10/11/2017, 4:41 PM
    maybe the key’d path could map to an 
    Address
    ? e.g. 
    3rdparty/bins/<name>/<version>:<name>
  • r

    rough-minister-58256

    10/11/2017, 4:42 PM
    then those would contain BUILD files mapping the name to a digest
  • w

    witty-crayon-22786

    10/11/2017, 4:42 PM
    the issue is that when a user wants to bump from version X to Y, needing to define ~12 targets would be a pain
  • w

    witty-crayon-22786

    10/11/2017, 4:42 PM
    as opposed to 
    name+version+binary_name
    + 
    Directory blob for manifest
  • a

    average-vr-56795

    10/11/2017, 4:43 PM
    I think the way Bazel solved this was by having a BUILD file basically packaged into the pex which would be implicitly mounted into the workspace somewhere
  • w

    witty-crayon-22786

    10/11/2017, 4:43 PM
    mm
  • a

    average-vr-56795

    10/11/2017, 4:44 PM
    But that's also a bit magic
  • w

    witty-crayon-22786

    10/11/2017, 4:45 PM
    the other magic bit is that users need to be able to sync these on-premise
  • w

    witty-crayon-22786

    10/11/2017, 4:45 PM
    currently that's cloning a listable thing
  • r

    rough-minister-58256

    10/11/2017, 4:45 PM
    could probably whip up a quick task impl that fetches and writes to some ingestable format
  • w

    witty-crayon-22786

    10/11/2017, 4:45 PM
    but it would be something like a sync process.
  • r

    rough-minister-58256

    10/11/2017, 4:45 PM
    or a task that did direct syncing
  • w

    witty-crayon-22786

    10/11/2017, 4:46 PM
    @rough-minister-58256: well, if it's just snapshot/`Directory` storage, cloning from X to Y CAS would be native i guess
  • w

    witty-crayon-22786

    10/11/2017, 4:46 PM
    right
  • r

    rough-minister-58256

    10/11/2017, 4:46 PM
    hmm ya
  • a

    average-vr-56795

    10/11/2017, 4:48 PM
    [FWIW I wasn't referring to a "grpc ContentAddressableStorage service" above, I was just referring to an S3 bucket where things are named according to their content digest in a flat namespace, but I guess they're analogous]
  • w

    witty-crayon-22786

    10/11/2017, 4:48 PM
    right
  • r

    rough-minister-58256

    10/11/2017, 4:48 PM
    I love the idea of proper snapshots as the container for these bins. then later its just a matter of snapshot composition to consume them
  • r

    rough-minister-58256

    10/11/2017, 4:48 PM
    vs fetch, create snapshot, compose
  • w

    witty-crayon-22786

    10/11/2017, 4:49 PM
    @average-vr-56795: and i think the use of the merkle tree / 
    Directory
    to define the manifest of artifacts available for a version is a pretty natural fit
  • r

    rough-minister-58256

    10/11/2017, 4:49 PM
    however that might present challenges in bootstrap cases?
  • w

    witty-crayon-22786

    10/11/2017, 4:49 PM
    @rough-minister-58256: we no longer fetch the engine... it's embedded. so there's that.
  • a

    average-vr-56795

    10/11/2017, 4:49 PM
    shrug there are a bunch of different wants I'd want to slice that information, and no particular Directory seems obviously like a winner, but sure
  • r

    rough-minister-58256

    10/11/2017, 4:50 PM
    was thinking more cmake et al
  • r

    rough-minister-58256

    10/11/2017, 4:50 PM
    but once we have John’s tool, prob a moot point
  • w

    witty-crayon-22786

    10/11/2017, 4:50 PM
    ...ah. those are definitely special casey, in that they should probably be in git.
1...353354355...906Latest