I'm experiencing massive slowdown when going from ...
# general
p
I'm experiencing massive slowdown when going from
2.16.0rc1
to
2.16.0.rc2
(or
2.16.0
) when running checks on BUILD files with
pants tailor --check update-build-files --check ::
- from checks of each file taking <1s to 5-25 seconds per file. I have parallelism set to
process_execution_local_parallelism=2
- problem goes away when this setting is changed to 1. This is a quite large repo, containing ~230 BUILD files.
1
interpreter_constraints = ["==3.8.*"]
we use this python version, pants runs on ubuntu 20.04
I tried to reproduce it on some example repos but with no luck, also I observed that when pants (black) checks subsequent files, first couple of files are processed quite fast, but after a couple of seconds it starts to block for 5-20seconds on each file (looks like it blocks on some resource)...
ok, looks like the issue goes away on
2.17.0rc1
...
I guess that's the way forward in that case 😄
h
Huh, would be good to pinpoint this
are you sure this is consistent?
I can't think of anything that changed between 2.16.0rc0 and rc1, and then changed back in 2.17.0rc1
p
@happy-kitchen-89482 I figured this out; I've re-read the docs https://www.pantsbuild.org/docs/python-lockfiles#lockfiles-for-tools and noticed there's a syntax change for requirements, I missed that earlier. After changing:
Copy code
[black]
interpreter_constraints = ["==3.8.*"]
install_from_resolve = "python-default"
to
Copy code
[black]
interpreter_constraints = ["==3.8.*"]
install_from_resolve = "python-default"
requirements = ["//3rdparty/python:requirements-dev#black"]
it works without problems on 2.16.0 - so I'm not sure what was the default value in rc1 vs rc2, but it was a configuration issue on my side after all.
h
Ah cool. Although - if you omit
requirements
it should still work, it'll just use the entire lockfile, which is probably not what you want.
So that would explain the slowdown
1
p
Makes sense, thanks!