https://pantsbuild.org/ logo
#development
Title
# development
a

ancient-vegetable-10556

06/11/2021, 8:15 PM
Hello! I’m trying to build Pants on an M1, and I’ve got the latest master, so even running under rosetta, it’s using the correct venv. Currently it looks like Rust is consistently building under the wrong architecture:
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % lipo -info /Users/chrisjrn/src/pants/src/python/pants/engine/internals/native_engine_pyo3.so
Non-fat file: /Users/chrisjrn/src/pants/src/python/pants/engine/internals/native_engine_pyo3.so is architecture: arm64
Continuing a discussion with @witty-crayon-22786
and I see that — even when I uninstall the arm toolchain from
rustup
, the boostrap process re-installs it:
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % MODE=debug ./pants                                                              
[=== 00:01 Building native engine... ===]
[=== 00:01 Building native_engine.so ===]
info: syncing channel updates for '1.52.1-aarch64-apple-darwin'
w

witty-crayon-22786

06/11/2021, 8:19 PM
your goal right now is to fully emulate?
a

ancient-vegetable-10556

06/11/2021, 8:19 PM
Right now, yes.
I can expand to running under arm architecture later
w

witty-crayon-22786

06/11/2021, 8:23 PM
ok. cc @hundreds-father-404 because he might be able to chime in on his strategy when he’s available.
a

ancient-vegetable-10556

06/11/2021, 8:23 PM
at the very least, making things work in a multi-arch situation like this will probably solve some problems for other people too
w

witty-crayon-22786

06/11/2021, 8:25 PM
and what is the current content of the
rust-toolchain
file? the
target
continues to seem very relevant
a

ancient-vegetable-10556

06/11/2021, 8:25 PM
Copy code
[toolchain]
channel = "1.52.1"
components = [
  "cargo",
  "clippy",
  "rust-std",
  "rustc",
  "rustfmt",
]
targets = [
  "x86_64-apple-darwin",
]
w

witty-crayon-22786

06/11/2021, 8:26 PM
ok, interesting. so maybe that doesn’t override the target that is explicitly passed to cargo.
a

ancient-vegetable-10556

06/11/2021, 8:26 PM
(surprise!)
This seems to come before cargo:
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % MODE=debug ./pants                                                              
[=== 00:01 Building native engine... ===]
[=== 00:01 Building native_engine.so ===]
info: syncing channel updates for '1.52.1-aarch64-apple-darwin'
info: latest update on 2021-05-10, rust version 1.52.1 (9bc8c42bb 2021-05-09)
info: downloading component 'cargo'
w

witty-crayon-22786

06/11/2021, 8:27 PM
you could try adding a:
Copy code
[build]
target = x86_64-apple-darwin
section around here: https://github.com/pantsbuild/pants/blob/main/src/rust/engine/Cargo.toml#L7
@ancient-vegetable-10556: re: “before cargo” yea… i expect that it’s going to actually run
cargo
using whatever it thinks the current platform/arch is
targets are for the compiled outputs of builds, rather than the build tool itself, i think.
so cargo being aarch is fine as long as cargo is using the relevant target/toolchain for the binary it is building
a

ancient-vegetable-10556

06/11/2021, 8:29 PM
Huh, so I think it matters that rustup was installed on my machine in arm mode
there’s probably somewhere else that we need to specify the arch
Copy code
warning: /Users/chrisjrn/src/pants/src/rust/engine/Cargo.toml: unused manifest key: build
w

witty-crayon-22786

06/11/2021, 8:30 PM
Huh, so I think it matters that rustup was installed on my machine in arm mode
it shouldn’t: that’s what i’m saying. i’m 98% sure i could use rustup/cargo compiled for my current platform to cross compile to a different platform.
a

ancient-vegetable-10556

06/11/2021, 8:30 PM
No, I mean from the perspective of the defaults that rustup uses
anyway, that build section was ignored
👍 1
and I got an
so
for the wrong arch
w

witty-crayon-22786

06/11/2021, 8:31 PM
for example: to cross compile for the browser, you run cargo as x86, but you set a “target” of
wasm-unknown-unknown
a

ancient-vegetable-10556

06/11/2021, 8:31 PM
Yeah, I get that. I think it’s impacting defaults
w

witty-crayon-22786

06/11/2021, 8:37 PM
looks like it needs to go in cargo “config” rather than the Cargo.toml: https://github.com/rust-lang/cargo/issues/4399 … so a
.cargo/config.toml
at the root of the repo
a

ancient-vegetable-10556

06/11/2021, 8:37 PM
let me look
w

witty-crayon-22786

06/11/2021, 8:37 PM
@ancient-vegetable-10556: also, to take out one variable here, you can ~directly invoke cargo with
./cargo
in the root of the repo
a

ancient-vegetable-10556

06/11/2021, 8:39 PM
it’s re-downloading a pile of deps, which seems promising
Copy code
Error: "Must provide PYO3_CROSS_LIB_DIR environment variable when cross-compiling"
warning: build failed, waiting for other jobs to finish...
it seems to think it’s cross-compiling something
w

witty-crayon-22786

06/11/2021, 8:52 PM
is the
arch …
set such that it should not think that?
a

ancient-vegetable-10556

06/11/2021, 8:52 PM
let me check
w

witty-crayon-22786

06/11/2021, 8:52 PM
my guess is that it detects “target is x, but actual platform is y”
a

ancient-vegetable-10556

06/11/2021, 8:53 PM
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % arch 
i386
chrisjrn@chrisjrns-MacBook-Pro pants % uname
Darwin
chrisjrn@chrisjrns-MacBook-Pro pants % uname -m
x86_64
chrisjrn@chrisjrns-MacBook-Pro pants %
w

witty-crayon-22786

06/11/2021, 8:54 PM
taking a step back, you made it further than this yesterday: you had gotten to plugin loading, which is well after this step (ie pants was running successfully, including native extensions). which arch were you targeting in that case?
a

ancient-vegetable-10556

06/11/2021, 8:54 PM
Oh, all of that was running under arm64
the problem was that there were python dependencies that weren’t available in arm64, so I switched to x86_64
w

witty-crayon-22786

06/11/2021, 8:55 PM
ok… given how much further along you were there, i wonder if continuing down that path and disabling plugins if need be wouldn’t be easier
right: but only plugins… can disable those as a workaround
a

ancient-vegetable-10556

06/11/2021, 8:56 PM
Allow me a few minutes to get back to where I was
w

witty-crayon-22786

06/11/2021, 8:56 PM
for example, to disable plugins for one run you could set
PANTS_PLUGINS='[]'
a

ancient-vegetable-10556

06/11/2021, 8:59 PM
it doesn’t like that 🙂
oh, I see why
yeah, that all worked, I was able to successfully bootstrap pants on arm
w

witty-crayon-22786

06/11/2021, 9:00 PM
ok, great. the only downside of disabling the TC plugin in Pants is that you won’t be able to use remote caching. we should figure that out in the next few weeks if we can, but if i had to guess, that is also where @hundreds-father-404 is currently.
a

ancient-vegetable-10556

06/11/2021, 9:01 PM
My hope is that we can figure out this Rust thing at some point
w

witty-crayon-22786

06/11/2021, 9:02 PM
yea, maybe. if you’re able to skip using Rosetta entirely, that’s good too.
a

ancient-vegetable-10556

06/11/2021, 9:02 PM
Right
that’s going to require dependencies compiled for the right arch
w

witty-crayon-22786

06/11/2021, 9:09 PM
so, the fact that Pants is running at all means that they are already compiled for the right arch, i think
a

ancient-vegetable-10556

06/11/2021, 9:10 PM
Not the plugins though, right?
w

witty-crayon-22786

06/11/2021, 9:10 PM
and only the plugins did not (hdrhistogram, toolchain)
correct
a

ancient-vegetable-10556

06/11/2021, 9:10 PM
Right
h

hundreds-father-404

06/11/2021, 11:28 PM
I don't have good reception to see this whole thread. But all I do is set PY=python3.9 and PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS to 3.9. I do not use Rosetta and the arch command. Rust builds an ARM binary
I use envrc to set both env vars
Oh and I don't need to disable TC plugin anymore, now that it works with Py39. I explicitly enable BuildSense and remote cache read and write
a

ancient-vegetable-10556

06/14/2021, 4:25 PM
OK, let’s see how that goes then
I should have clicked that I needed python 3.9 enabled
Looks like I already had Python 3.9 enabled, and the thing that was failing was the
hdrhistogram
plugin. This, on the other hand, affects the toolchain plugin:
Copy code
Needed macosx_11_0_arm64-cp-39-cp39 compatible dependencies for:
 1: cryptography>=3.2
    Required by:
      pyOpenSSL 20.0.1
    But this pex had no 'cryptography' distributions.
 2: cryptography>=1.3.4; extra == "security"
    Required by:
      requests 2.25.1
    But this pex had no 'cryptography' distributions.
h

hundreds-father-404

06/14/2021, 4:33 PM
Oh. Run with --no-process-execution-local-cache as an experiment
a

ancient-vegetable-10556

06/14/2021, 4:33 PM
that didn’t help
h

hundreds-father-404

06/14/2021, 4:34 PM
Hm. Is there more to the error? What command are you running?
a

ancient-vegetable-10556

06/14/2021, 4:35 PM
I’m running
MODE=debug  ./pants --no-process-execution-local-cache
; I have
hdrhistogram
commented out in my config file, and there’s one more line that I missed (excluding the traceback)
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /private/var/folders/0m/h2n902qn38b8555xq8z0w_h40000gn/T/process-executionSOzQ5z/.tmp/tmpkbur2u36.
h

hundreds-father-404

06/14/2021, 4:36 PM
What's the traceback?
a

ancient-vegetable-10556

06/14/2021, 4:37 PM
Copy code
File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 484, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 540, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 656, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 688, in execute_pkg_resources
    return runner()
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/bin/pex.py", line 1161, in main
    seed_info = seed_cache(options, pex, verbose=options.seed == Seed.VERBOSE)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/bin/pex.py", line 1215, in seed_cache
    venv_pex = ensure_venv(pex)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/pex_bootstrapper.py", line 421, in ensure_venv
    shenbang = populate_venv_with_pex(
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/tools/commands/venv.py", line 117, in populate_venv_with_pex
    for dist in pex.resolve():
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/pex.py", line 118, in resolve
    for dist in env.resolve():
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/environment.py", line 608, in resolve
    self._resolved_dists = self.resolve_dists(all_reqs)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/environment.py", line 692, in resolve_dists
    raise ResolveError(
h

hundreds-father-404

06/14/2021, 4:41 PM
Try renaming or rm rfing ~/.cache/pants. What's happening is the cache has x86 platform in it and now you're using arm64
a

ancient-vegetable-10556

06/14/2021, 4:42 PM
fwiw, we merged a fix last week that makes the venvs in the cache be arch-specific (https://github.com/pantsbuild/pants/pull/12192). Is there somewhere else in
.cache
that is arch-specific?
(noting that removing the cache did indeed work)
w

witty-crayon-22786

06/14/2021, 4:45 PM
there is actually… i’ll post a fix for that.
💯 1
a

ancient-vegetable-10556

06/14/2021, 4:45 PM
thank you!
h

hundreds-father-404

06/14/2021, 4:46 PM
@witty-crayon-22786 the fix is to finish my design doc to make platform be arch aware, right?
w

witty-crayon-22786

06/14/2021, 4:47 PM
uhm. mayhaps. my fix may have nothing to do with the above.
ah. so the issue above is that we’re not including the full platform/arch when invoking PEX?
h

hundreds-father-404

06/14/2021, 4:51 PM
Yeah, the resolve requirements Process stored in lmdb_store keeps OS in mind, but not arch. If we fixed that, the subsequent Pex Process should choose the correct repository pex. I think the fix is with Pants, not Pex Idk Chris's roadmap, but this would be an excellent introduction to Pants, including a little work on rust. I could write up instructions and/or video chat
w

witty-crayon-22786

06/14/2021, 4:52 PM
possibly, yea. we paired on https://github.com/pantsbuild/pants/issues/12059 last week, so a good next thing maybe.
👍 1