https://pantsbuild.org/ logo
Powered by
#pex
  • r

    rough-minister-58256

    10/09/2019, 9:07 PM
    we had a process that would unpack resources from pex files via a special util entrypoint that used python’s zip lib; tl;dr, it took ~5+ minutes to unpack a handful of tiny text files via this route and on the order of ~5 seconds to do the same with 
    zip(1)
    a
    • 2
    • 1
  • a

    aloof-angle-91616

    10/09/2019, 9:12 PM
    there's also https://github.com/indygreg/PyOxidizer which imports from memory using rust
  • a

    aloof-angle-91616

    10/09/2019, 9:13 PM
    i really want to pursue packaging pants with that
  • a

    aloof-angle-91616

    10/09/2019, 9:13 PM
    was at a meetup at facebook with the guy talking about it, super cool
  • c

    cuddly-window-48195

    10/10/2019, 7:52 PM
    FYI, I filed a ticket to expose the has of a given target; @aloof-angle-91616 suggested I share it here: https://github.com/pantsbuild/pants/issues/8445
  • r

    rough-minister-58256

    10/10/2019, 9:00 PM
    anyone have any current advice for how best to achieve pex-style env scrubbing via API? we want to bootstrap arbitrary pex environments in a jupyter notebook kernel w/ env scrubbing for hermeticity.
    e
    • 2
    • 19
  • e

    enough-analyst-54434

    10/17/2019, 8:18 PM
  • e

    enough-analyst-54434

    10/20/2019, 10:12 PM
    The idea of vendoring pip in pex to gain some pip features came up recently. I've started an issue here to discuss the idea: https://github.com/pantsbuild/pex/issues/781 @rough-minister-58256, @hundreds-father-404, @dry-analyst-73584, @happy-kitchen-89482 and others interested, I'd appreciate your considerations here. There is some care needed to keep built pexes small (we shouldn't need all of pip inside every built PEX for runtime use), but this may be otherwise sane. I do worry I'm missing problems here and counting on you all to spot them. Thanks in advance for your brain power.
    📝 1
    👌 1
    👍 1
    h
    • 2
    • 2
  • e

    early-needle-54791

    10/21/2019, 6:05 PM
    Hey @enough-analyst-54434 I opened https://github.com/pantsbuild/pants/issues/8502 to track an issue with pants release. It seems the gist of it is that pex sees different manylinux platform tags as incompatible, for instance a manylinux2010 whl cannot fulfill a dep for a manylinux1 whl. I'm coming up with a workaround for the short term, but I'm wondering if this is something we plan to fix in pex upstream.
    a
    w
    +2
    • 5
    • 9
  • a

    aloof-angle-91616

    10/21/2019, 7:04 PM
    https://pantsbuild.slack.com/archives/C046T6T9U/p1571672725116000 is this a known issue for some wheels? do we know if there is a workaround? cc @cuddly-window-48195
  • c

    calm-artist-46894

    11/01/2019, 12:12 AM
    Hi, I opened https://github.com/pantsbuild/pex/issues/785 to track an issue where user environment leaked into pex virtual environment. When pex constructs the minimum_sys_modules, it only drops tainted modules which is a package (with path) but keeps the ones which doesn't have path. This will leave some residual from user's environment into the pex virtual environment, for example modules like 
    Jinja2.utils
    . Also submitted a PR for possible fix: https://github.com/pantsbuild/pex/pull/786/files Please help taking a look
  • e

    enough-analyst-54434

    11/10/2019, 5:43 PM
    If you use 
    pex --platform
    or the equivalent 
    python_binary.platforms
    BUILD configuration from Pants, please consider reading and chiming in on https://github.com/pantsbuild/pex/issues/781#issuecomment-552215761 TLDR: I'd like to remove pex support for inexact platforms like 
    --platform=linux-x86_64
    and require exact platforms like 
    --platform=linux-x86_64-cp-27-mu
    . If inexact platforms like 
    linux-x86_64
    are desired, I'd like to move support for those up into Pants or local Pants customizations through target/target macro plugins.
    👌 1
    👍 3
    r
    • 2
    • 14
  • r

    rough-minister-58256

    11/12/2019, 7:55 PM
    @enough-analyst-54434 FYI, saw this on Twitter yesterday - it might help better inform the long-term decision around vendoring pip as resolver: https://twitter.com/di_codes/status/1193980331004743680 + https://twitter.com/di_codes/status/1193980341549178880
    🔥 2
  • r

    rough-minister-58256

    11/12/2019, 7:55 PM
    (note the thread there, these are just the two salient points within)
  • h

    happy-kitchen-89482

    11/13/2019, 6:19 PM
    I should mention that we (Toolchain) have our own resolver algorithm, currently implemented as a service. So this provides some competition to pip in that space.
  • h

    happy-kitchen-89482

    11/13/2019, 6:20 PM
    It's a language-agnostic implementation of https://medium.com/@nex3/pubgrub-2fb6470504f
  • h

    happy-kitchen-89482

    11/13/2019, 6:21 PM
    We should probably open-source it
    💯 3
    ✍️ 1
    👨‍🎓 1
    🔥 1
    💵 1
  • a

    aloof-angle-91616

    11/13/2019, 6:24 PM
    i didn't realize that was something you'd be able to open-source, i would be extremely interested in that, that's super super cool
  • a

    aloof-angle-91616

    11/13/2019, 6:24 PM
    @dry-analyst-73584's work right?
  • d

    dry-analyst-73584

    11/13/2019, 6:30 PM
    Yep
  • r

    rough-minister-58256

    11/13/2019, 9:11 PM
    nice!
  • r

    rough-minister-58256

    11/13/2019, 9:11 PM
    I’d love for 3rdparty resolution to be more pluggable and composable, generally speaking.
  • r

    rough-minister-58256

    11/13/2019, 9:12 PM
    I think pex could make sense as a suite of components, e.g. 
    pex_resolve -r requirements.txt | pex_package -o pex_runtime && ./pex_runtime
  • r

    rough-minister-58256

    11/13/2019, 9:13 PM
    the packaging and runtime bits could generalize to support more than just e.g. python apps in the long term
  • r

    rough-minister-58256

    11/13/2019, 9:14 PM
    this could support smarter app runtimes that could do things like bootstrap their own resources/deps/interpreters/etc or other interesting features
    a
    h
    • 3
    • 11
  • r

    rough-minister-58256

    11/13/2019, 9:15 PM
    but the resolve aspect to me should be made pluggable vs keying into any one particular impl
  • r

    rough-minister-58256

    11/13/2019, 9:15 PM
    so pex, pip, toolchain can all compose
  • a

    aloof-angle-91616

    11/13/2019, 9:16 PM
    there’s also pants 
    remote_sources()
    which currently is used to extract native libs out of tensorflow and protobuf out of jvm libs
  • a

    aloof-angle-91616

    11/13/2019, 9:19 PM
    and things like coursier have specific methods on top of normal jvm resolution to support building things as scala-native or native-image. which is relevant to other tools who may want to invoke jvm tools as binaries. may be relevant.
  • e

    enough-analyst-54434

    11/13/2019, 11:41 PM
1...626364...74Latest