https://pantsbuild.org/ logo
Join Slack
Powered by
Can anyone recommend a large (but not "huge") pyth...
# development
w
Can anyone recommend a large (but not "huge") python library that I can install without doing any chicanery on my local system, and that has rock-solid wheels? e.g. not pytorch - I'm using this so far. Pretty good, takes about 10 seconds on my system - would like to get it to 30 to really tell what's going on
Copy code
fastapi
numpy
pandas
pydantic
scipy
uvicorn
I'm writing a sanity test to be able to script a few performance scenarios while building pexes/dockerfiles/etc
b
Tensorflow has been simpler for me than pytorch in the past (at least, just for installing, not running)
w
I need to do a superficial run, just to make sure I've "proved" they're working. e.g.
Copy code
# Meaningless comment that I'll use to test re-builds    

import os

import fastapi
import numpy
import pandas
import pydantic
import scipy
import uvicorn

MY_ID = "sj"
MY_ENV_ID = os.getenv("PERFANITY_ID", "env")

class Foo(pydantic.BaseModel):
    id: str  

app = fastapi.FastAPI()

@app.get("/")
async def root():
    foo = Foo(id=MY_ID)
    bar = Foo(id=MY_ENV_ID)
    return {"message": f"Hello {foo} and {bar}"}


def square(x: int) -> int:
    return x**2

a = numpy.mean([1,2,3])
b = pandas.DataFrame()
c = scipy.integrate.quad(square, 0, 4)

if __name__ == "__main__":
    uvicorn.run(app)
n
pyspark?
c
Do you need large wheels or just a large total? 
azure-cli
pulls in a full gig of stuff with ~150 packages
w
Large wheels that I can hello-world. I may have found the sweet spot for dev, with about 7-10 seconds of deps building, since I'm running like... 30 tests 🙂 ("experiments")
High level - need to extract the timing for these scenarios, and then add multi-stage docker, multiple pexes, etc
Copy code
for layout in loose packed zipapp; do
        for execution_mode in venv zipapp; do
            echo "  Packaging pex with execution_mode=$execution_mode and layout=$layout"
            time pants package simple:bin@execution_mode=$execution_mode,layout=$layout
            echo "  Running tests with zero changes"
            time pants test simple:

            echo "  Re-packaging with zero changes"
            time pants package simple:bin@execution_mode=$execution_mode,layout=$layout
            echo "  Re-running tests with zero changes"
            time pants test simple:

            echo "  Re-packaging with a comment change"
            sed -i '' "1s|.*|# CACHE_BUSTING=$RANDOM|" ./simple/main.py
            time pants package simple:bin@execution_mode=$execution_mode,layout=$layout
            echo "  Re-running tests with zero changes"
            time pants test simple:
        done
    done
b
You could consider the 
--no-pre-install-wheels
pex option too (I think there pants fields to pass through arbitrary args), although this’ll be affected by the pex cache. For truly fresh-cache tests, I find setting the local store/named cache global options to temporary dirs to be the most reliable way.
w
Yeah, I haven't decided the shape of the tests just yet - right now just building up the concept, looking for obvious flaws/wins. In the variant above, it assumes a pre-built cache to start - and I'm more curious about the incremental change. I have another set for multi-stage docker builds too, and then working on one for multiple pexes in docker, and so on - ideally looking to built up an ala carte set of macros for different scenarios
So, this is the idea of what I'm working on (https://github.com/sureshjoshi/perfanity). Not sure of the shape exactly - but something like this - just a long set of experiments with durations based goals and whatnot. Looking at how to nicely chart file count and file sizes - but can only get so multi-dimensional while still being decently exportable to a CSV. The purpose is to be a companion to Josh's awesome pex/docker optimization blog post. https://www.pantsbuild.org/blog/2022/08/02/optimizing-python-docker-deploys-using-pants Investigating adding in more dimensions somehow (different interpreters, different pants versions, etc) - but that's all advancement type of stuff. NOTE: The "clean" aren't actually clean, because they still pull in "some" cache - as I haven't implemented a cache clean. And there is still inter-run coupling which is no bueno
Copy code
step                                             clean               noop           incremental           
goal                                           package   run test package  run test     package   run test
simple:bin@execution_mode=venv,layout=loose       4.02 11.29 1.11    1.00 3.63 0.09        4.10 11.03 1.11
simple:bin@execution_mode=venv,layout=packed      1.23  3.02 0.09    0.09 1.08 0.25        1.17  1.71 1.10
simple:bin@execution_mode=venv,layout=zipapp      6.38  0.94 0.08    0.08 0.89 0.08        6.40  1.51 1.08
simple:bin@execution_mode=zipapp,layout=loose     4.23  9.02 0.09    0.96 8.93 0.09        4.19  9.45 1.04
simple:bin@execution_mode=zipapp,layout=packed    1.15  1.36 0.08    0.08 1.20 0.09        1.15  1.26 1.08
simple:bin@execution_mode=zipapp,layout=zipapp    6.35  1.30 0.08    0.08 1.38 0.16        6.43  1.30 1.06
Note to self: Don't use 
loose
packaging when copying into Docker 🤦‍♂️ Though, it looks like it's not even using my pre-built - so, who knows - I probably blundered the cache somewhere
Copy code
step                                             clean                noop             incremental            
goal                                           package    run test package    run test     package    run test
simple:img_execution_mode_venv_layout_loose       6.25 354.93 0.18    5.87 355.49 0.20        5.74 357.60 0.19
simple:img_execution_mode_venv_layout_packed      4.45  14.58 0.18    3.91  15.06 0.19        3.52  14.66 0.42
simple:img_execution_mode_venv_layout_zipapp      4.32  15.52 0.19    3.67  15.62 0.21        3.82  14.83 0.18
simple:img_execution_mode_zipapp_layout_loose     6.15 360.92 0.21    6.01 357.35 0.18        5.79 358.86 0.20
simple:img_execution_mode_zipapp_layout_packed    4.46  14.63 0.21    3.88  14.88 0.22        4.22  15.67 0.24
simple:img_execution_mode_zipapp_layout_zipapp    4.13  15.83 0.35    3.59  14.95 0.29        3.51  15.39 0.21
b
If you're looking for ideas, I wonder if that might be file IO overheads again: taking a long time to serialize/load/manipulate the docker context. Are you running on a macOS machine? I think there's a way to increase the docker build logging verbosity to see what it says.
w
Yeah, that one was likely the same problem if I recall. I think it was on WSL actually, which is like, the WORST file I/O on my old mac mini. I use it as my worst case scenario half the time (not in a Windows mount, so it's particularly egregious). The plan is to run in Github CI a bit, working on optimizing that a bit with caching in some places - but it's still rough there. The Docker context is definitely a big slowdown
c
Note to self: Don't use loose packaging when copying into Docker
https://github.com/pantsbuild/pants/issues/20822
w
Yeah, I didn't spend more than a few seconds digging into it - I want to get the whole pipeline setup and running, then I can revisit. The initial purpose of the repo was more "what's going on with the permutations", the "why" will come much later on
e.g. I have locally some using GH action cache, GH registry cache, etc
6 Views
Open in Slack
PreviousNext
Powered by