https://pantsbuild.org/ logo
Powered by
#general
  • b

    bored-art-40741

    03/16/2016, 11:20 PM
    e.g. in a CI workspace
  • b

    bored-art-40741

    03/16/2016, 11:20 PM
    Then clean it up every time as an exit hook
  • r

    rough-minister-58256

    03/16/2016, 11:21 PM
    you can override the pex root at runtime like: 
    PEX_ROOT=/alternative/dir ./the.pex
  • m

    millions-controller-72965

    03/16/2016, 11:22 PM
    how does it work currently if say supervisor starts up 4 instances of the same pex binary?
  • m

    millions-controller-72965

    03/16/2016, 11:22 PM
    do they all write to the same cache folder?
  • r

    rough-minister-58256

    03/16/2016, 11:22 PM
    yup
  • m

    mammoth-pharmacist-54215

    03/16/2016, 11:22 PM
    yes
  • m

    millions-controller-72965

    03/16/2016, 11:22 PM
    so they're pretty much stepping over each other's files? potentially writing the same file up to 4 times?
  • m

    mammoth-pharmacist-54215

    03/16/2016, 11:22 PM
    its a little ugly right now we have an open issue for at least pulling it out of your home dir in pants. We have a PR in pex to enable that.
  • r

    rough-minister-58256

    03/16/2016, 11:23 PM
    well, there’d prob be a race but the theory would be one pex writes teh cache and the other 3 just reuse it
  • m

    mammoth-pharmacist-54215

    03/16/2016, 11:23 PM
    more importantly right now if one process corrupts a file it potentially impacts other processes which is why we want to fix it.
  • r

    rough-minister-58256

    03/16/2016, 11:24 PM
    @mammoth-pharmacist-54215 he means at runtime - e.g. to support zip_safe=False, the pex runtime splats its contents out to PEX_ROOT
  • m

    mammoth-pharmacist-54215

    03/16/2016, 11:24 PM
    oh you mean with the binary created by pants
  • m

    mammoth-pharmacist-54215

    03/16/2016, 11:24 PM
    gotcha
  • m

    millions-controller-72965

    03/16/2016, 11:24 PM
    interesting; sounds like the best way currently would be to give each instance a unique PEX_ROOT and then that way it can be cleaned up on exit?
  • r

    rough-minister-58256

    03/16/2016, 11:25 PM
    @millions-controller-72965: yeah, that sounds like the best current way
  • m

    mammoth-pharmacist-54215

    03/16/2016, 11:25 PM
    yeah, that would probably be the safest
  • m

    millions-controller-72965

    03/16/2016, 11:25 PM
    negates the benefits of the cache but probably the cleanest way so far
  • r

    rough-minister-58256

    03/16/2016, 11:25 PM
    depends how you construct the PEX_ROOT value, but yeah
  • m

    millions-controller-72965

    03/16/2016, 11:26 PM
    how do you guys typically handle static files/templates?
  • m

    millions-controller-72965

    03/16/2016, 11:26 PM
    if there's a way to get those working w/o zip_safe=False, then all of these temp files don't become an issue right?
  • m

    millions-controller-72965

    03/16/2016, 11:26 PM
    i'm using the tornado framework, and w/o zip_safe=False, all the template dirs etc breaks
  • r

    rough-minister-58256

    03/16/2016, 11:27 PM
    you basically just need to implement a quick content loader for whatevers loading the resources that understands how to pull things out of the zip - 
    pkg_resources.resource_stream
    is popular
  • m

    millions-controller-72965

    03/16/2016, 11:28 PM
    ok thanks!
  • r

    rough-minister-58256

    03/16/2016, 11:29 PM
    another approach would be to have some intermediate hook that splats all of the resources out to disk
  • r

    rough-minister-58256

    03/16/2016, 11:29 PM
    then you point your native content loader to that new dir outside of the pex
  • m

    millions-controller-72965

    03/16/2016, 11:30 PM
    yea but that would pretty much have the same cleanup issues as zip_safe=False
  • r

    rough-minister-58256

    03/16/2016, 11:30 PM
    yeah
  • m

    millions-controller-72965

    03/16/2016, 11:31 PM
    any interest in a feature in pex that would self cleanup upon exit? if so i could do a PR for that
  • m

    millions-controller-72965

    03/16/2016, 11:31 PM
    it would be a flag like zip_safe; pretty much the conclusion from above, negating the caching functionality
1...106010611062...2420Latest