https://pantsbuild.org/ logo
#general
Title
# general
w

worried-painter-31382

09/26/2022, 3:34 PM
Hi! We still have trouble with running pants in a python subprocess. Some new findings: 1. It looks like
pantsd
isn't being re-used. We get
Copy code
17:26:09.72 [INFO] Initializing scheduler...
17:26:09.85 [INFO] Scheduler initialized.
repeatedly for subprocesses. 2. It looks like python2.7 is used for some pex unzip thingy? Is this expected? 3. We're having cache misses everytime the subprocess is rerun. Could it be due to pyenv, XDG_CACHE_HOME_DIR or something else? It's hard to figure out what settings causes pants to disregard its cache.
b

bitter-ability-32190

09/26/2022, 3:41 PM
For 1. It's useful to check
.pants.d/pants.log
to see what
inotify
is telling pants has changed
✔️ 1
w

worried-painter-31382

09/26/2022, 3:42 PM
Ill just add that the subprocesses are ran serially
b

bitter-ability-32190

09/26/2022, 3:42 PM
For 3. Running with
-ldebug
and looking for "Running Process..." Log lines and comparing like-elements across runs can help identify what inputs have changed
✔️ 1
h

happy-kitchen-89482

09/26/2022, 4:12 PM
To clarify, you’re spawning pants itself from python, using
subprocess
or similar?
And re 2. can you post the output that mentions 2.7?
b

bitter-ability-32190

09/26/2022, 4:42 PM
Actually for 1, the log helps. But if a file changes the daemon doesn't restart. I was misremembering. Usually I've seen this happen if you exceed the memory cache limits
w

worried-painter-31382

09/26/2022, 4:52 PM
Exactly right @happy-kitchen-89482. Timezones wont let me check in like 14 hrs 😅 but Ill post here ASAP regarding the pex unzip python2.7 thing
h

happy-kitchen-89482

09/26/2022, 4:58 PM
Hmm, and you don’t have that same problem of the daemon restarting if you run
./pants
directly from the shell?
I’ve tried to naively reproduce using
subprocess
in a repl, and I don’t see the issue happening
w

worried-painter-31382

09/26/2022, 5:02 PM
I'll see if I can reproduce from memory how our machines are setup, but my uneducated suspicion is the subprocess losing pyenv and cache dir
And no, pantsd issues are isolated to subprocess
b

bitter-ability-32190

09/26/2022, 5:10 PM
Is the cwd the same across subprocess runs? Pants stores the daemon PID relative to cwd
👀 1
w

worried-painter-31382

09/26/2022, 5:14 PM
That is a good tip!
-ldebug
for the different invocations,
app.py
runs
Copy code
subprocess.run(
       [str(pants_path), '-ldebug', 'package', str(dockerfile), '--dynamic-ui=False'],
       text=True,
)
Copy code
./pants package <snip>
20:08:06.55 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f8c4aae53a0>
20:08:06.55 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f8c4aae53a0>
20:08:06.55 [DEBUG] Connecting to pantsd on port 43245
20:08:06.55 [DEBUG] Connecting to pantsd on port 43245 attempt 1/3
20:08:06.55 [DEBUG] Connected to pantsd
20:08:06.57 [DEBUG] Launching 1 roots (poll=false).
20:08:06.57 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.
20:08:06.58 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
20:08:06.58 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.scope_options()) of Some("@rule(construct_scope_())") changed.
20:08:06.58 [DEBUG] Dependency @rule(construct_scope_()) of Some("@rule(pants.init.plugin_resolver.resolve_plugins())") changed.
20:08:06.58 [DEBUG] computed 1 nodes in 0.011746 seconds. there are 7 total nodes.
20:08:06.60 [INFO] Initialization options changed: reinitializing scheduler...
Copy code
$ python app.py
20:11:17.27 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f1555c3f580>
20:11:17.27 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f1555c3f580>
20:11:17.27 [DEBUG] Connecting to pantsd on port 43245
20:11:17.27 [DEBUG] Connecting to pantsd on port 43245 attempt 1/3
20:11:17.27 [DEBUG] masking tcgetattr exception: error(25, 'Inappropriate ioctl for device')
20:11:17.28 [DEBUG] Connected to pantsd
20:11:17.31 [DEBUG] Launching 1 roots (poll=false).
20:11:17.31 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.
20:11:17.32 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
20:11:17.32 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.scope_options()) of Some("@rule(construct_scope_())") changed.
20:11:17.32 [DEBUG] Dependency @rule(construct_scope_()) of Some("@rule(pants.init.plugin_resolver.resolve_plugins())") changed.
20:11:17.32 [DEBUG] computed 1 nodes in 0.010312 seconds. there are 7 total nodes.
20:11:17.35 [INFO] Initialization options changed: reinitializing scheduler...
b

bitter-ability-32190

09/26/2022, 6:25 PM
I don't see any process runs in there 👀
w

worried-painter-31382

09/26/2022, 6:46 PM
I though the issue would materialize from the decisions pants is doing? "Initialization options changed: reinitializing schedule" seems to me like pants has decided to restart the daemon (and it invalidates the cache) based on some options? Im very curious what causes it to for example think the plugins have changed between these invocations (notice the
20:11:17.32 [DEBUG] Dependency @rule(construct_scope_()) of Some("@rule(pants.init.plugin_resolver.resolve_plugins())") changed.
)
b

bitter-ability-32190

09/26/2022, 7:32 PM
What's your plugin list look like?
w

worried-painter-31382

09/26/2022, 7:32 PM
Copy code
[GLOBAL]
pythonpath = ["%(buildroot)s/pants-plugins"]

pants_version = "2.13.0"
backend_packages = [
    "pants.backend.python",
    "pants.backend.docker",
    "hash",
]


[source]
root_patterns = ["/"]
b

bitter-ability-32190

09/26/2022, 7:34 PM
Nothing screams at me, but I'm less adept here 🫤
@witty-crayon-22786?
w

worried-painter-31382

09/26/2022, 7:36 PM
hmm. There's a
__pycache__
directory in the plugin folder
b

bitter-ability-32190

09/26/2022, 7:36 PM
That doesn't seem unexpected
The code does get executed from source
w

worried-painter-31382

09/26/2022, 7:36 PM
I guess not, was unsure how
pants_ignore
worked but read it again, it's gitignore
removing our custom plugins changed nothing
Copy code
python app.py                                                                                                                                           21:40:59.89 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f471b174eb0>                                                                         21:40:59.90 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f471b174eb0>                                                                         21:40:59.90 [DEBUG] Connecting to pantsd on port 35825                                                                                                                                     21:40:59.90 [DEBUG] Connecting to pantsd on port 35825 attempt 1/3                                                                                                                         21:40:59.90 [DEBUG] Connected to pantsd                                                                                                                                                    21:40:59.93 [DEBUG] Launching 1 roots (poll=false).                                                                                                                                        21:40:59.94 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.                                                             21:40:59.95 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             21:40:59.95 [DEBUG] computed 1 nodes in 0.010659 seconds. there are 7 total nodes.                                                                                                         21:40:59.97 [DEBUG] Launching 1 roots (poll=false).                                                                                                                                        21:40:59.97 [DEBUG] computed 1 nodes in 0.000277 seconds. there are 7 total nodes.                                                                                                         21:41:00.00 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(FileLiteralSpec(file='cloud/deployment/jwk_cacher/Dockerfile'),), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False))                                   21:41:00.00 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)                                                               21:41:00.00 [DEBUG] Launching 1 roots (poll=false).                                                                                                                                        21:41:00.00 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.                                                             21:41:00.00 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             21:41:00.00 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             21:41:00.01 [DEBUG] computed 1 nodes in 0.001878 seconds. there are 818 total nodes.                                                                                                       21:41:00.01 [DEBUG] requesting <class 'pants.backend.project_info.list_targets.List'> to satisfy execution of `list` goal                                                                  21:41:00.01 [DEBUG] Launching 1 roots (poll=false).                                                                                                                                        21:41:00.01 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             21:41:00.01 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             21:41:00.01 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             21:41:00.01 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.             cloud/deployment/jwk_cacher:jwk-cacher                                                                                                                                                     21:41:00.01 [DEBUG] Completed: `list` goal                                                                                                                                                 21:41:00.01 [DEBUG] computed 1 nodes in 0.007508 seconds. there are 820 total nodes.
Copy code
./pants list -ldebug /home/tobias/projects/profiler/cloud/deployment/jwk_cacher/Dockerfile
21:43:40.79 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f2d66691e20>
21:43:40.79 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f2d66691e20>
21:43:40.79 [DEBUG] Connecting to pantsd on port 35825
21:43:40.80 [DEBUG] Connecting to pantsd on port 35825 attempt 1/3
21:43:40.80 [DEBUG] Connected to pantsd
21:43:40.81 [DEBUG] Launching 1 roots (poll=false).
21:43:40.81 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.
21:43:40.82 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
21:43:40.82 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.scope_options()) of Some("@rule(construct_scope_())") changed.
21:43:40.82 [DEBUG] Dependency @rule(construct_scope_()) of Some("@rule(pants.init.plugin_resolver.resolve_plugins())") changed.
21:43:40.82 [DEBUG] computed 1 nodes in 0.014019 seconds. there are 7 total nodes.
21:43:40.85 [INFO] Initialization options changed: reinitializing scheduler...
21:43:40.85 [DEBUG] terminating pantsd service: <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7eee74368dc0>
21:43:41.27 [DEBUG] terminating pantsd service: <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7eee74365040>
21:43:41.27 [DEBUG] File handle limit is: 10000
21:43:41.27 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: nailgun::CommandRunner { inner: local::CommandRunner { .. }, .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: nailgun::CommandRunner { inner: local::CommandRunner { .. }, .. }, .. }, .. }] for process execution.
21:43:41.37 [DEBUG] Changes to /home/tobias/.cache/pants/setup/bootstrap-Linux-x86_64/pants.QA6V74/install/bin, outside of the buildroot, will not be invalidated.
21:43:41.37 [DEBUG] Changes to /home/tobias/.cache/pants/setup/bootstrap-Linux-x86_64/pants.QA6V74/install/bin, outside of the buildroot, will not be invalidated.
21:43:41.37 [DEBUG] Changes to /home/tobias/.pyenv/versions/3.9.13/lib/python39.zip, outside of the buildroot, will not be invalidated.
21:43:41.37 [DEBUG] Changes to /home/tobias/.pyenv/versions/3.9.13/lib/python3.9, outside of the buildroot, will not be invalidated.
21:43:41.37 [DEBUG] Changes to /home/tobias/.pyenv/versions/3.9.13/lib/python3.9/lib-dynload, outside of the buildroot, will not be invalidated.
21:43:41.37 [DEBUG] Changes to /home/tobias/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages, outside of the buildroot, will not be invalidated.
21:43:41.37 [DEBUG] setting up service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7eecce188820>
21:43:41.37 [DEBUG] setting up service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7eecce188a60>
21:43:41.37 [DEBUG] starting service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7eecce188820>
21:43:41.38 [DEBUG] starting service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7eecce188a60>
21:43:41.38 [INFO] Scheduler initialized.
21:43:41.38 [DEBUG] Launching 1 roots (poll=false).
21:43:41.38 [DEBUG] computed 1 nodes in 0.000405 seconds. there are 7 total nodes.
21:43:41.42 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(FileLiteralSpec(file='cloud/deployment/jwk_cacher/Dockerfile'),), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False))
21:43:41.42 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
21:43:41.42 [DEBUG] Launching 1 roots (poll=false).
21:43:41.42 [DEBUG] computed 1 nodes in 0.002089 seconds. there are 17 total nodes.
21:43:41.42 [DEBUG] requesting <class 'pants.backend.project_info.list_targets.List'> to satisfy execution of `list` goal
21:43:41.42 [DEBUG] Launching 1 roots (poll=false).
21:43:41.42 [DEBUG] Completed: Find targets from input specs
21:43:41.56 [DEBUG] Completed: Generate `python_requirement` targets from requirements.txt
21:43:41.72 [DEBUG] Completed: Find targets from input specs
21:43:41.72 [DEBUG] Completed: Find targets from input specs
cloud/deployment/jwk_cacher:jwk-cacher
21:43:41.72 [DEBUG] Completed: `list` goal
21:43:41.72 [DEBUG] computed 1 nodes in 0.298842 seconds. there are 818 total nodes.
full runs, just running the
list
goal for some target, once via bash and once via python subprocess
I still dont understand what is causing pants to not like the pantsd that is running, but have to restart it everytime python subprocess is ran
b

bitter-ability-32190

09/26/2022, 7:55 PM
We'll hold on, when you run the list command twice it should reuse the daemon. Passing -ldebug changes the bootstrap options so restarts the daemon
I'm also on mobile so I might've missed something
w

witty-crayon-22786

09/26/2022, 7:58 PM
Copy code
20:08:06.60 [INFO] Initialization options changed: reinitializing scheduler...
suggests that a bootstrap /
daemon=True
option (an even smaller set of options) has changed between runs
if the CLI args are all consistent (the most likely culprit), and you’re not changing
pants.toml
, the next thing to check would just be whether some env vars are changing between runs.
as Josh said: a run with
-ldebug
and then another without will cause the scheduler to invalidate, because `--level`/`-l` are bootstrap options
so, yea: would look at the CLI args in each run, and then probably the env in each run, and look for differences in which option values are being set.
w

worried-painter-31382

09/26/2022, 8:03 PM
./pants list requirements.txt -ldebug && python -c "import subprocess; subprocess.run(['/home/tobias/projects/profiler/pants', 'list', 'requirements.txt', '-ldebug'])"
Copy code
22:02:16.94 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7fcc41e96cd0>
22:02:16.94 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7fcc41e96cd0>
22:02:16.94 [DEBUG] Connecting to pantsd on port 38351
22:02:16.95 [DEBUG] Connecting to pantsd on port 38351 attempt 1/3
22:02:16.95 [DEBUG] Connected to pantsd
22:02:16.96 [DEBUG] Launching 1 roots (poll=false).
22:02:16.96 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.
22:02:16.97 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:16.97 [DEBUG] computed 1 nodes in 0.011350 seconds. there are 7 total nodes.
22:02:17.00 [DEBUG] Launching 1 roots (poll=false).
22:02:17.00 [DEBUG] computed 1 nodes in 0.000408 seconds. there are 7 total nodes.
22:02:17.03 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(FileLiteralSpec(file='requirements.txt'),), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False))
22:02:17.03 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
22:02:17.03 [DEBUG] Launching 1 roots (poll=false).
22:02:17.03 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.
22:02:17.03 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:17.03 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:17.03 [DEBUG] computed 1 nodes in 0.002203 seconds. there are 747 total nodes.
22:02:17.03 [DEBUG] requesting <class 'pants.backend.project_info.list_targets.List'> to satisfy execution of `list` goal
22:02:17.03 [DEBUG] Launching 1 roots (poll=false).
22:02:17.03 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:17.03 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:17.03 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:17.04 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
//:requirements_file
//requirements.txt:requirements
22:02:17.04 [DEBUG] Completed: `list` goal
22:02:17.04 [DEBUG] computed 1 nodes in 0.007651 seconds. there are 749 total nodes.
22:02:17.95 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f8fb8e81070>
22:02:17.95 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x7f8fb8e81070>
22:02:17.95 [DEBUG] Connecting to pantsd on port 38351
22:02:17.95 [DEBUG] Connecting to pantsd on port 38351 attempt 1/3
22:02:17.95 [DEBUG] Connected to pantsd
22:02:17.96 [DEBUG] Launching 1 roots (poll=false).
22:02:17.96 [DEBUG] Dependency SessionValues of Some("@rule(pants.engine.internals.options_parsing.parse_options())") changed.
22:02:17.97 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.parse_options()) of Some("@rule(pants.engine.internals.options_parsing.scope_options())") changed.
22:02:17.97 [DEBUG] Dependency @rule(pants.engine.internals.options_parsing.scope_options()) of Some("@rule(construct_scope_())") changed.
22:02:17.97 [DEBUG] Dependency @rule(construct_scope_()) of Some("@rule(pants.init.plugin_resolver.resolve_plugins())") changed.
22:02:17.97 [DEBUG] computed 1 nodes in 0.011941 seconds. there are 7 total nodes.
22:02:18.00 [INFO] Initialization options changed: reinitializing scheduler...
22:02:18.00 [DEBUG] terminating pantsd service: <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7fadef40ea60>
22:02:18.13 [DEBUG] terminating pantsd service: <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7fadef097d60>
22:02:18.14 [DEBUG] File handle limit is: 10000
22:02:18.14 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: nailgun::CommandRunner { inner: local::CommandRunner { .. }, .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: nailgun::CommandRunner { inner: local::CommandRunner { .. }, .. }, .. }, .. }] for process execution.
22:02:18.24 [DEBUG] Changes to /home/tobias/.cache/pants/setup/bootstrap-Linux-x86_64/pants.QA6V74/install/bin, outside of the buildroot, will not be invalidated.
22:02:18.24 [DEBUG] Changes to /home/tobias/.cache/pants/setup/bootstrap-Linux-x86_64/pants.QA6V74/install/bin, outside of the buildroot, will not be invalidated.
22:02:18.24 [DEBUG] Changes to /home/tobias/.pyenv/versions/3.9.13/lib/python39.zip, outside of the buildroot, will not be invalidated.
22:02:18.24 [DEBUG] Changes to /home/tobias/.pyenv/versions/3.9.13/lib/python3.9, outside of the buildroot, will not be invalidated.
22:02:18.24 [DEBUG] Changes to /home/tobias/.pyenv/versions/3.9.13/lib/python3.9/lib-dynload, outside of the buildroot, will not be invalidated.
22:02:18.24 [DEBUG] Changes to /home/tobias/.cache/pants/setup/bootstrap-Linux-x86_64/2.13.0_py39/lib/python3.9/site-packages, outside of the buildroot, will not be invalidated.
22:02:18.24 [DEBUG] setting up service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7f6ac0e80cd0>
22:02:18.24 [DEBUG] setting up service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7f6ac0e80f10>
22:02:18.24 [DEBUG] starting service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x7f6ac0e80cd0>
22:02:18.24 [DEBUG] starting service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x7f6ac0e80f10>
22:02:18.24 [INFO] Scheduler initialized.
22:02:18.24 [DEBUG] Launching 1 roots (poll=false).
22:02:18.24 [DEBUG] computed 1 nodes in 0.000525 seconds. there are 7 total nodes.
22:02:18.28 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(FileLiteralSpec(file='requirements.txt'),), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False))
22:02:18.28 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
22:02:18.28 [DEBUG] Launching 1 roots (poll=false).
22:02:18.28 [DEBUG] computed 1 nodes in 0.002242 seconds. there are 22 total nodes.
22:02:18.28 [DEBUG] requesting <class 'pants.backend.project_info.list_targets.List'> to satisfy execution of `list` goal
22:02:18.28 [DEBUG] Launching 1 roots (poll=false).
22:02:18.29 [DEBUG] Completed: Find targets from input specs
22:02:18.44 [DEBUG] Completed: Generate `python_requirement` targets from requirements.txt
22:02:18.53 [DEBUG] Completed: Find targets from input specs
22:02:18.53 [DEBUG] Completed: Find targets from input specs
//:requirements_file
//requirements.txt:requirements
22:02:18.53 [DEBUG] Completed: `list` goal
22:02:18.54 [DEBUG] computed 1 nodes in 0.251346 seconds. there are 743 total nodes.
So it must be env variables then?
w

witty-crayon-22786

09/26/2022, 8:04 PM
seems likely, yea. whether the output is a TTY should no longer invalidate the scheduler
and to be clear: this is about your question 1, although env vars can definitely also impact your question 3
✔️ 1
w

worried-painter-31382

09/26/2022, 8:22 PM
Would a different PATH cause this? I notice pyenv shims append stuff to path before entering python
w

witty-crayon-22786

09/26/2022, 8:22 PM
yes, a different PATH would definitely cause 3
it should not cause 1
w

worried-painter-31382

09/26/2022, 9:04 PM
it should not cause 1
Seems it doesn't, I've circumvented pyenv and it still happens
I've reduced the whole thing to pants.toml
Copy code
[GLOBAL]
pants_version = "2.13.0"

backend_packages = [
    "pants.backend.python"
]
BUILD
Copy code
```
src/main.py
```print("Hello world!")
src/BUILD
Copy code
python_sources()
Just to sanity check, it still occurs that the daemon is restarted when running
./pants list src/main.py -ldebug && python -c "import subprocess; subprocess.run(['/home/tobias/projects/dummy/pants', 'list', 'src/main.py', '-ldebug'])"
Is there some way I can render the options pants is detecting to change between these two?
w

witty-crayon-22786

09/26/2022, 9:15 PM
could dump out and diff
./pants help-all
(tons of JSON), although note that there is apparently some non-determinism in there, which shouldn’t impact anything aside from making your diff noisy
👍 1
w

worried-painter-31382

09/26/2022, 9:25 PM
Copy code
{
                                "details": "from env var PANTS_BIN_NAME",
                                "rank": "ENVIRONMENT",
                                "value": "/home/tobias/projects/dummy/pants"
                            },
Copy code
{
                                "details": "from env var PANTS_BIN_NAME",
                                "rank": "ENVIRONMENT",
                                "value": "./pants"
                            },
b

bitter-ability-32190

09/26/2022, 9:27 PM
Ah yeah that makes sense. It's the difference between invoking the wrapper script and invoking the code directly. FWIW you can set that in pants.toml. see if that helps
w

witty-crayon-22786

09/26/2022, 9:28 PM
…that could indeed do it. all bootstrap options are fingerprinted, unless they opt out with `fingerprint=False`: https://github.com/pantsbuild/pants/blob/9a6dc0069bcd56e0bfd1b4445ff40674b99d41ae/src/python/pants/option/global_options.py#L774-L779
w

worried-painter-31382

09/26/2022, 9:29 PM
It does seem to be the culprit 😅
Ah yeah that makes sense. It's the difference between invoking the wrapper script and invoking the code directly.
FWIW you can set that in pants.toml. see if that helps
The script invocation seem to override it unfortunately
Buut I can work with this, as long as
cwd
of the subprocess is repo root
./pants
should be ok, and we already estabillished cwd should be repo root anyway because of pantsd dirs
b

bitter-ability-32190

09/26/2022, 9:47 PM
If you unconditionally set it to "./pants" does that "work"? The script overrides it with the same value
w

worried-painter-31382

09/26/2022, 9:49 PM
If you unconditionally set it to "./pants" does that "work"?
If you mean in
pants.toml
the awnser is no
b

bitter-ability-32190

09/26/2022, 10:18 PM
Hm shame
w

worried-painter-31382

09/27/2022, 1:41 PM
Just wanted to reconnect here. The bin name var was part of the reason for restarts, the other was pantsd running out of its 1GiB memory.
The python2.7 thing was a pyenv vs system python issue
h

happy-kitchen-89482

09/27/2022, 2:01 PM
Thanks for the update. So are things working now?
b

bitter-ability-32190

09/27/2022, 2:02 PM
Have you found the setting to change the daemon memory size? Look in the global options (sorry on mobile)
5 Views