Hi again! Whenever a run pants, there’s a fair amo...
# generall
lively-gpu-26436
08/07/2023, 8:44 AMHi again! Whenever a run pants, there’s a fair amount of time between the moment I submit the command and the moment Pants starts emitting output (I’m using Pants 2.16.0). Is there I way I look at what Pants is doing during this time to try to optimize it? I’m also using MacOS but in Linux is taking around the same amount of time.
✅ 1
b
boundless-salesmen-11323
08/07/2023, 8:46 AMDo you set no_pantsd somewhere? This looks a lot like the scheduler is not initialized which leads to higher startup times.
l
lively-gpu-26436
08/07/2023, 10:00 AMno I know of, this is my entire
pants.toml Copy code
[GLOBAL]
pants_version = "2.16.0"
concurrent = true
backend_packages = [
"pants.backend.python",
"pants.backend.experimental.python",
"pants.backend.experimental.python.lint.ruff",
"pants.backend.python.lint.black",
"pants.backend.python.lint.docformatter",
"pants.backend.python.typecheck.mypy",
]
[source]
root_patterns = ["/", "/src", "/bin", "/3rdparty", "/mypy-stubs"]
[generate-lockfiles]
diff = true
[python]
enable_resolves = true
interpreter_constraints = ["==3.11.4"]
default_run_goal_use_sandbox = false
[python.resolves]
python-default = "3rdparty/requirements.lock"
[pytest]
args = ["--maxfail=3"]
install_from_resolve = "python-default"
xdist_enabled = false
requirements = ["pytest-xdist", "future", "pytest", "faker"]
[black]
interpreter_constraints = ["==3.11.4"]
install_from_resolve = "python-default"
requirements = ["black"]
[ruff]
interpreter_constraints = ["==3.11.4"]
install_from_resolve = "python-default"
requirements = ["ruff"]
[mypy]
interpreter_constraints = ["==3.11.4"]
install_from_resolve = "python-default"
requirements = ["mypy", "pydantic"]
config = "pyproject.toml"
config_discovery = false
[docformatter]
interpreter_constraints = ["==3.11.4"]
install_from_resolve = "python-default"
requirements = ["docformatter"]
[setuptools]
install_from_resolve = "pants-tools"
requirements = ["setuptools", "wheel"]
[repl]
shell = "ipython"
[ipython]
install_from_resolve = "python-default"
requirements = ["ipython"]
[python-bootstrap]
search_path = [
"<PATH>",
"/usr/local/bin",
]
[anonymous-telemetry]
enabled = falselively-gpu-26436
08/07/2023, 10:30 AMI just found about the debug flag, so running it with that I can see most of the time is spent in the “using cache” lines
Copy code
python-exp(main): pants -ldebug test ::
12:28:06.32 [DEBUG] File handle limit is: 10000
→ 12:28:06.33 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }] for process execution.
12:28:09.85 [DEBUG] Launching 1 roots (poll=false).
12:28:09.86 [DEBUG] computed 1 nodes in 0.007775 seconds. there are 9 total nodes.
12:28:09.86 [DEBUG] Launching 1 roots (poll=false).
12:28:09.86 [DEBUG] computed 1 nodes in 0.000137 seconds. there are 13 total nodes.
12:28:09.93 [DEBUG] File handle limit is: 10000
→ 12:28:09.94 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }] for process execution.
12:28:13.90 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(RecursiveGlobSpec(directory=''),), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False))
12:28:13.90 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependents=<DependentsOption.NONE: 'none'>)
12:28:13.90 [DEBUG] Launching 1 roots (poll=false).b
broad-processor-92400
08/07/2023, 10:41 AMI think this might be caused by
concurrent = true--no-pantsdpants --concurrent …PANTS_CONCURRENT=true pants …l
lively-gpu-26436
08/07/2023, 10:44 AMthat indeed improved things a lot, thanks @broad-processor-92400!
🎉 1