seeing this one a clean master checkout ```$ pants goals Bui Pants #development

seeing this one a clean master checkout: ```$ ./pa...

wide-energy-11069

04/09/2021, 9:34 PM

seeing this one a clean master checkout:

Copy code

$ ./pants goals
Building native engine
error: invalid channel name '[toolchain]' in '/Users/yic/workspace/pants_2/rust-toolchain'
info: caused by: invalid toolchain name: '[toolchain]'

Failed to build native engine.

any advice?

hundreds-father-404

04/09/2021, 9:35 PM

your rustup is out of date. Try

rustup update

wide-energy-11069

04/09/2021, 9:36 PM

oh so pants no longer bootstraps one there?

hundreds-father-404

04/09/2021, 9:36 PM

No we don't, which simplified a lot of our setup. We've been trying to chip away at all our custom tooling

wide-energy-11069

04/09/2021, 9:37 PM

ok. still seeing the same issue after

Copy code

$ rustup update
info: syncing channel updates for 'stable-x86_64-apple-darwin'
sdfinfo: latest update on 2021-03-25, rust version 1.51.0 (2fd73fabe 2021-03-23)
^R
sdinfo: downloading component 'rustc'
^R
Total:  56.6 MiB Speed:  10.6 MiB/s                
info: downloading component 'rust-std'
Total:  23.6 MiB Speed:  10.7 MiB/s                
info: downloading component 'cargo'
info: downloading component 'rust-docs'
Total:  15.0 MiB Speed:  10.1 MiB/s                
info: downloading component 'rust-src'
info: removing component 'rustc'
info: removing component 'rust-std'
info: removing component 'cargo'
info: removing component 'rust-docs'
info: removing component 'rust-src'
info: installing component 'rustc'
info: installing component 'rust-std'
info: installing component 'cargo'
info: installing component 'rust-docs'
info: installing component 'rust-src'
info: checking for self-updates
info: downloading self-update

  stable-x86_64-apple-darwin updated - rustc 1.51.0 (2fd73fabe 2021-03-23)

average-vr-56795

04/09/2021, 9:38 PM

rustup update self

average-vr-56795

04/09/2021, 9:38 PM

Oops,

rustup self update

🙌 1

average-vr-56795

04/09/2021, 9:38 PM

That way around 😄

wide-energy-11069

04/09/2021, 9:39 PM

Copy code

[tw-mbp-yic pants_2 (master)]$ rustup self update
info: checking for self-updates
info: downloading self-update
info: rustup updated successfully to 1.23.1
[tw-mbp-yic pants_2 (master)]$ 
[tw-mbp-yic pants_2 (master)]$ 
[tw-mbp-yic pants_2 (master)]$ rustup update
info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: checking for self-updates
info: downloading self-update

  stable-x86_64-apple-darwin unchanged - rustc 1.51.0 (2fd73fabe 2021-03-23)

[tw-mbp-yic pants_2 (master)]$ 
[tw-mbp-yic pants_2 (master)]$ ./pants goals
Building native engine
error: invalid channel name '[toolchain]' in '/Users/yic/workspace/pants_2/rust-toolchain'
info: caused by: invalid toolchain name: '[toolchain]'

Failed to build native engine.

wide-energy-11069

04/09/2021, 9:40 PM

does this matter?

Copy code

$ env | grep RUS
RUSTUP_HOME=/Users/yic/.cache/pants/rust/rustup

wide-energy-11069

04/09/2021, 9:40 PM

unsetting doesn’t help either

hundreds-father-404

04/09/2021, 9:41 PM

what does

which -a rustup

say? FYI

./cargo

is the script that we use which invokes your global

cargo

binary in

bootstrap_code.sh

average-vr-56795

04/09/2021, 9:46 PM

I'd probably

rm -rf ~/.cache/pants/rust

👍 1

wide-energy-11069

04/09/2021, 9:47 PM

haha not so fast, we still need to for 1.25.x-twtr occasionally

😞 1

wide-energy-11069

04/09/2021, 9:47 PM

Copy code

[tw-mbp-yic pants_2 (master)]$ which -a rustup
/Users/yic/.cargo/bin/rustup
[tw-mbp-yic pants_2 (master)]$ which -a cargo
/Users/yic/.cargo/bin/cargo
[tw-mbp-yic pants_2 (master)]$ /Users/yic/.cargo/bin/rustup --version
rustup 1.7.0 (813f7b7a8 2017-10-30)
[tw-mbp-yic pants_2 (master)]$ /Users/yic/.cargo/bin/rustup self update
info: checking for self-updates
info: downloading self-update
info: rustup updated successfully to 1.23.1
[tw-mbp-yic pants_2 (master)]$ /Users/yic/.cargo/bin/rustup --version
rustup 1.7.0 (813f7b7a8 2017-10-30)

👀 1

wide-energy-11069

04/09/2021, 9:48 PM

this is weird, it says it’s updated to 1.23.1, then version shows again it’s on 1.7.0

hundreds-father-404

04/09/2021, 9:48 PM

Weird. Maybe reinstall it: https://rustup.rs

👌 1

average-vr-56795

04/09/2021, 9:49 PM

Copy code

% rustup --version
rustup 1.23.1 (3df2264a9 2020-11-30)

wide-energy-11069

04/09/2021, 9:50 PM

ok reinstall did the trick. thanks folks!

❤️ 1

wide-energy-11069

04/09/2021, 10:12 PM

also been at

Copy code

Compiling testutil v0.0.1 (/Users/yic/workspace/pants_2/src/rust/engine/testutil)
   Compiling mock v0.0.1 (/Users/yic/workspace/pants_2/src/rust/engine/testutil/mock)
   Compiling process_execution v0.0.1 (/Users/yic/workspace/pants_2/src/rust/engine/process_execution)
    Finished dev [unoptimized + debuginfo] target(s) in 3m 56s
14:54:35.02 [INFO] initializing scheduler...
14:56:05.11 [INFO] Long running tasks:
  88.23s 	Find interpreter for constraints: CPython==3.7.4
14:56:35.14 [INFO] Long running tasks:
  118.27s 	Find interpreter for constraints: CPython==3.7.4
...
15:10:36.44 [INFO] Long running tasks:
  959.57s 	Find interpreter for constraints: CPython==3.7.4
15:11:06.45 [INFO] Long running tasks:
  989.58s 	Find interpreter for constraints: CPython==3.7.4

for a while. let me know if i need to kill it. python3.7 has been 100%

👀 1

hundreds-father-404

04/09/2021, 10:13 PM

Uh..that is not expected. If you'd be willing, it would be helpful if you could do

--no-process-execution-local-cleanup

, find the tmpdir for that process, and run it's

__run.sh

to see if it hangs. We'd love to get a hold of that PEX You can

ctrl-c

wide-energy-11069

04/09/2021, 10:21 PM

Copy code

502 40173 40168   0  3:17PM ??         1:22.53 /Users/yic/.cache/pants/pants_dev_deps/Darwin.py37.venv/bin/python3.7 /private/var/folders/yt/pxdd3gp96530xh_116kskmkh0000gp/T/process-execution4GBs1T/.tmp/tmpa20ikulx -c import hashlib, os, sys\012\012python = os.path.realpath(sys.executable)\012print(python)\012\012hasher = hashlib.sha256()\012with open(python, "rb") as fp:\012  for chunk in iter(lambda: fp.read(8192), b""):\012      hasher.update(chunk)\012print(hasher.hexdigest())\012

Copy code

$ cd /private/var/folders/yt/pxdd3gp96530xh_116kskmkh0000gp/T/process-execution4GBs1T
$ find . | grep run.sh
<nothing>

is this what you mean?

hundreds-father-404

04/09/2021, 10:24 PM

I mean https://www.pantsbuild.org/v2.5/docs/troubleshooting#debug-tip-inspect-the-chroot-with---no-process-execution-local-cleanup

wide-energy-11069

04/09/2021, 10:27 PM

oh ok, in that case I haven’t hit that point yet.

Copy code

$ ./pants --no-process-execution-local-cleanup goals
15:17:57.43 [INFO] initializing scheduler...
...
15:25:28.08 [INFO] Long running tasks:
  449.97s 	Find interpreter for constraints: CPython==3.7.4
15:25:58.10 [INFO] Long running tasks:
  479.98s 	Find interpreter for constraints: CPython==3.7.4

wide-energy-11069

04/09/2021, 10:28 PM

based on the process, it wants to do some hashing?

Copy code

502 40173 40168   0  3:17PM ??         9:50.16 /Users/yic/.cache/pants/pants_dev_deps/Darwin.py37.venv/bin/python3.7 /private/var/folders/yt/pxdd3gp96530xh_116kskmkh0000gp/T/process-execution4GBs1T/.tmp/tmpa20ikulx -c import hashlib, os, sys\012\012python = os.path.realpath(sys.executable)\012print(python)\012\012hasher = hashlib.sha256()\012with open(python, "rb") as fp:\012  for chunk in iter(lambda: fp.read(8192), b""):\012      hasher.update(chunk)\012print(hasher.hexdigest())\012

hundreds-father-404

04/09/2021, 10:28 PM

Hm it should have dumped the log path at the very beginning of starting the process, you don't see that higher up? 🤔

wide-energy-11069

04/09/2021, 10:29 PM

full:

Copy code

$ ./pants --no-process-execution-local-cleanup goals
15:17:57.43 [INFO] initializing scheduler...
15:19:27.54 [INFO] Long running tasks:
  89.43s 	Find interpreter for constraints: CPython==3.7.4
15:19:57.57 [INFO] Long running tasks:
  119.45s 	Find interpreter for constraints: CPython==3.7.4
15:20:27.60 [INFO] Long running tasks:
  149.48s 	Find interpreter for constraints: CPython==3.7.4
15:20:57.64 [INFO] Long running tasks:
  179.52s 	Find interpreter for constraints: CPython==3.7.4
15:21:27.69 [INFO] Long running tasks:
  209.57s 	Find interpreter for constraints: CPython==3.7.4
15:21:57.75 [INFO] Long running tasks:
  239.64s 	Find interpreter for constraints: CPython==3.7.4
15:22:27.80 [INFO] Long running tasks:
  269.69s 	Find interpreter for constraints: CPython==3.7.4
15:22:57.80 [INFO] Long running tasks:
  299.69s 	Find interpreter for constraints: CPython==3.7.4
15:23:27.81 [INFO] Long running tasks:
  329.70s 	Find interpreter for constraints: CPython==3.7.4
15:23:57.91 [INFO] Long running tasks:
  359.80s 	Find interpreter for constraints: CPython==3.7.4
15:24:27.99 [INFO] Long running tasks:
  389.88s 	Find interpreter for constraints: CPython==3.7.4
15:24:58.07 [INFO] Long running tasks:
  419.95s 	Find interpreter for constraints: CPython==3.7.4
15:25:28.08 [INFO] Long running tasks:
  449.97s 	Find interpreter for constraints: CPython==3.7.4
15:25:58.10 [INFO] Long running tasks:
  479.98s 	Find interpreter for constraints: CPython==3.7.4
15:26:28.18 [INFO] Long running tasks:
  510.07s 	Find interpreter for constraints: CPython==3.7.4
15:26:58.21 [INFO] Long running tasks:
  540.10s 	Find interpreter for constraints: CPython==3.7.4
15:27:28.31 [INFO] Long running tasks:
  570.19s 	Find interpreter for constraints: CPython==3.7.4
15:27:58.39 [INFO] Long running tasks:
  600.28s 	Find interpreter for constraints: CPython==3.7.4
15:28:28.47 [INFO] Long running tasks:
  630.36s 	Find interpreter for constraints: CPython==3.7.4

hundreds-father-404

04/09/2021, 10:29 PM

Oh, try

--no-pantsd

maybe

wide-energy-11069

04/09/2021, 10:31 PM

ok. this is the stacktrace when I killed it in case that’s useful:

Copy code

Engine traceback:
  in select
  in pants.init.plugin_resolver.resolve_plugins (CPython==3.7.4)
  in pants.backend.python.util_rules.pex.create_venv_pex (pants_plugins.pex)
  in pants.backend.python.util_rules.pex.build_pex (pants_plugins.pex)
  in pants.backend.python.util_rules.pex.find_interpreter (CPython==3.7.4)
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/Users/yic/workspace/pants_2/src/python/pants/engine/process.py", line 255, in fallible_to_exec_result_or_raise
    description.value,
pants.engine.process.ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython==3.7.4' failed with exit code 241.
stdout:

stderr:

Traceback (most recent call last):
  File "/Users/yic/workspace/pants_2/src/python/pants/bin/daemon_pants_runner.py", line 125, in single_daemonized_run
    scheduler, options_initializer = self._core.prepare(options_bootstrapper, complete_env)
  File "/Users/yic/workspace/pants_2/src/python/pants/pantsd/pants_daemon_core.py", line 129, in prepare
    self._initialize(options_fingerprint, options_bootstrapper, env)
  File "/Users/yic/workspace/pants_2/src/python/pants/pantsd/pants_daemon_core.py", line 104, in _initialize
    raise e
  File "/Users/yic/workspace/pants_2/src/python/pants/pantsd/pants_daemon_core.py", line 90, in _initialize
    options_bootstrapper, env, raise_=True
  File "/Users/yic/workspace/pants_2/src/python/pants/init/options_initializer.py", line 113, in build_config_and_options
    self._plugin_resolver, options_bootstrapper, env
  File "/Users/yic/workspace/pants_2/src/python/pants/init/options_initializer.py", line 44, in _initialize_build_configuration
    working_set = plugin_resolver.resolve(options_bootstrapper, env)
  File "/Users/yic/workspace/pants_2/src/python/pants/init/plugin_resolver.py", line 114, in resolve
    for resolved_plugin_location in self._resolve_plugins(options_bootstrapper, env):
  File "/Users/yic/workspace/pants_2/src/python/pants/init/plugin_resolver.py", line 137, in _resolve_plugins
    session.product_request(ResolvedPluginDistributions, [self._interpreter_constraints])[
  File "/Users/yic/workspace/pants_2/src/python/pants/engine/internals/scheduler.py", line 536, in product_request
    self._raise_on_error([t for _, t in throws])
  File "/Users/yic/workspace/pants_2/src/python/pants/engine/internals/scheduler.py", line 476, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

Engine traceback:
  in select
  in pants.init.plugin_resolver.resolve_plugins (CPython==3.7.4)
  in pants.backend.python.util_rules.pex.create_venv_pex (pants_plugins.pex)
  in pants.backend.python.util_rules.pex.build_pex (pants_plugins.pex)
  in pants.backend.python.util_rules.pex.find_interpreter (CPython==3.7.4)
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/Users/yic/workspace/pants_2/src/python/pants/engine/process.py", line 255, in fallible_to_exec_result_or_raise
    description.value,
pants.engine.process.ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython==3.7.4' failed with exit code 241.
stdout:

stderr:

👍 1

hundreds-father-404

04/09/2021, 10:36 PM

Btw the process shouldn't take more than 0.5s, maybe 3s in a weird case

wide-energy-11069

04/09/2021, 10:36 PM

no dice. yeah, will need to debug this a bit. purely from the process it wants to do:

Copy code

import hashlib, os, sys

python = os.path.realpath(sys.executable)
print(python)

hasher = hashlib.sha256()
with open(python, "rb") as fp:
  for chunk in iter(lambda: fp.read(8192), b""):
      hasher.update(chunk)
print(hasher.hexdigest())

let me see it’s because of symlink cycles and what not.

happy-kitchen-89482

04/10/2021, 12:25 AM

@wide-energy-11069 any luck?

wide-energy-11069

04/10/2021, 12:26 AM

still working on it, but put up the diff: https://pantsbuild.slack.com/archives/G0107QFGGAW/p1618014341018400?thread_ts=1617988608.015700&cid=G0107QFGGAW

wide-energy-11069

04/10/2021, 12:35 AM

linux box works, so i’ll just go with that

Open in Slack

Previous Next