https://pantsbuild.org/ logo
Join Slack
Powered by
Difference in execution times of running check He...
# general
f
Difference in execution times of running check Hey. I'm just starting with pants and I am seeing 
pants check ::
is much slower (~10 seconds) than running the type checker (
pyright
) directly (~1.5 seconds).
Copy code
$ time pants check ::
08:49:58.47 [INFO] Canceled: Force venv to materialize
08:49:58.71 [INFO] Completed: Force venv to materialize
08:50:09.19 [INFO] Completed: Run Pyright on 8 files.
08:50:09.19 [ERROR] Completed: Typecheck using Pyright - pyright - pyright failed (exit code 1).
calculations/adder.py
  calculations/adder.py:3:5 - error: "x" is not defined (reportUndefinedVariable)
1 error, 0 warnings, 0 informations



āœ• pyright failed.
pants check ::  0.01s user 0.03s system 0% cpu 11.013 total
Copy code
āÆ time pyright
calculations/adder.py
  calculations/adder.py:3:5 - error: "x" is not defined (reportUndefinedVariable)
1 error, 0 warnings, 0 informations
pyright  0.59s user 0.14s system 55% cpu 1.309 total
Is this to be expected? Am I missing something around caching or setup? šŸ¤” Any advise greatly appreciated
h
Hmmm, does this happen every time you re-run 
pants check ::
after a failure?
Just checking that it's not some one-time setup thing
f
Hey @happy-kitchen-89482. Yes, it is happening every time after a failure. When I terminate the 
pantsd
background process it takes ~25 seconds for the first run. Then ~10 seconds every time after that.
h
Ouch
How long does pyright take on a first run after you nuke its cache?
f
~3 seconds
Copy code
$ time pyright

added 2 packages, and audited 3 packages in 1s

found 0 vulnerabilities
calculations/adder.py
  calculations/adder.py:3:5 - error: "t" is not defined (reportUndefinedVariable)
1 error, 0 warnings, 0 informations
pyright  1.90s user 1.01s system 95% cpu 3.056 total
Interestingly 
pants check ::
was slightly faster after removing the pyright cache šŸ˜…
Copy code
time pants check ::
20:00:53.07 [INFO] Canceled: Force venv to materialize
20:00:53.72 [INFO] Completed: Force venv to materialize
20:01:01.43 [INFO] Completed: Run Pyright on 8 files.
20:01:01.43 [ERROR] Completed: Typecheck using Pyright - pyright - pyright failed (exit code 1).
calculations/adder.py
  calculations/adder.py:3:5 - error: "t" is not defined (reportUndefinedVariable)
1 error, 0 warnings, 0 informations



āœ• pyright failed.
pants check ::  0.01s user 0.01s system 0% cpu 8.569 total
w
Can you run with 
-ldebug
? I'm curious if any of the intermediate steps between forcing the venv to materialize, and the process running are in the logs - we might be missing a caching step in there. Also, if you keep the sandbox and run the pyright script directly from there, is it any faster?
On my absolutely trivial sanity-type check, I see that running 
npx pyright ...
is about 2x faster than 
pants check ::
(not 1:1, but definitely close enough for this) Running it via the sandbox is just under 2x slower - but I also noticed that by default we're using node 16? And my system is on node 22 or something, I believe there must be some sort of improvements there. Also, via npx I'm using 
pyright 1.1.360
and pants is using 
1.1.316
by default
Bumping to the same v22 nodejs version brought the time differential to about 20% slower
Copy code
[nodejs]
version = "v22.0.0"
f
It doesn't spend long on the 
Force venv to materialize
step. It completes almost immediately and then I get the following timer:
Copy code
20:32:22.00 [INFO] Completed: Force venv to materialize
ā ˆ 2.76s Run Pyright on 8 files.
I'm not getting any extra output with 
-ldebug
šŸ¤”
Copy code
āÆ time pants -ldebug check ::
20:34:01.01 [INFO] Completed: Force venv to materialize
20:34:07.71 [INFO] Completed: Run Pyright on 8 files.
20:34:07.71 [ERROR] Completed: Typecheck using Pyright - pyright - pyright failed (exit code 1).
calculations/adder.py
  calculations/adder.py:3:5 - error: "t" is not defined (reportUndefinedVariable)
1 error, 0 warnings, 0 informations



āœ• pyright failed.
pants -ldebug check ::  0.01s user 0.01s system 0% cpu 7.180 total
Also, if you keep the sandbox and run the pyright script directly from there, is it any faster?
I'm afraid I'm new to pants. How should I do this?
The number of files in this project is tiny btw.
Copy code
$ find . -name '*.py' | wc -l
       8
It's just a small test project to play around with pants
w
time pants --keep-sandboxes=on_failure check ::
With debugging (pretty verbose) 
time pants -ldebug --keep-sandboxes=on_failure check ::
That'll give you something like 
13:25:23.12 [INFO] Preserving local process execution dir /private/var/folders/dy/q08y_dts5vd71rm99t4gc9lr0000gp/T/pants-sandbox-aFigFs for Run Pyright on 5 files.
Then you can cd into the directory, and run 
sh __run.sh
The number of files in this project is tiny btw
Yep, saw that, I made an equally small repo at like 3-5 files - I'm seeing 1-2 seconds, so 10s is blowing my mind. šŸ¤Æ Even in my trivial case with the 20% overhead, I still think that's too much for a re-run of what should be fully cached info.
Anyways, if you're able to run that test by updating the nodejs version in your pants.toml to give a second point of information: pants.toml:
Copy code
[nodejs]
version = "v22.0.0"
Even if you don't keep it like that, it'll help figure out what the perf causes might be. Also, I do think Pants should bump the defaults anyways
Actually, interestingly, after updating Node - the 
pyright
execution time on my machine is almost the same as running it via 
npx
- probably within run-to-run variance. The rest of the time is pants itself going through rule selection and stuff... šŸ¤· Using
Copy code
[GLOBAL]
pants_version = "2.19.1"
7 Views
Open in Slack
PreviousNext
Powered by