https://pantsbuild.org/ logo
Join Slack
Powered by
Hello! I’m trying to build Pants on an M1, and I’v...
# development
a
Hello! I’m trying to build Pants on an M1, and I’ve got the latest master, so even running under rosetta, it’s using the correct venv. Currently it looks like Rust is consistently building under the wrong architecture:
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % lipo -info /Users/chrisjrn/src/pants/src/python/pants/engine/internals/native_engine_pyo3.so
Non-fat file: /Users/chrisjrn/src/pants/src/python/pants/engine/internals/native_engine_pyo3.so is architecture: arm64
Continuing a discussion with @witty-crayon-22786
and I see that — even when I uninstall the arm toolchain from 
rustup
, the boostrap process re-installs it:
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % MODE=debug ./pants                                                              
[=== 00:01 Building native engine... ===]
[=== 00:01 Building native_engine.so ===]
info: syncing channel updates for '1.52.1-aarch64-apple-darwin'
w
your goal right now is to fully emulate?
a
Right now, yes.
I can expand to running under arm architecture later
w
ok. cc @hundreds-father-404 because he might be able to chime in on his strategy when he’s available.
a
at the very least, making things work in a multi-arch situation like this will probably solve some problems for other people too
w
and what is the current content of the 
rust-toolchain
file? the 
target
continues to seem very relevant
a
Copy code
[toolchain]
channel = "1.52.1"
components = [
  "cargo",
  "clippy",
  "rust-std",
  "rustc",
  "rustfmt",
]
targets = [
  "x86_64-apple-darwin",
]
w
ok, interesting. so maybe that doesn’t override the target that is explicitly passed to cargo.
a
(surprise!)
This seems to come before cargo:
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % MODE=debug ./pants                                                              
[=== 00:01 Building native engine... ===]
[=== 00:01 Building native_engine.so ===]
info: syncing channel updates for '1.52.1-aarch64-apple-darwin'
info: latest update on 2021-05-10, rust version 1.52.1 (9bc8c42bb 2021-05-09)
info: downloading component 'cargo'
w
you could try adding a:
Copy code
[build]
target = x86_64-apple-darwin
section around here: https://github.com/pantsbuild/pants/blob/main/src/rust/engine/Cargo.toml#L7
@ancient-vegetable-10556: re: “before cargo” yea… i expect that it’s going to actually run 
cargo
using whatever it thinks the current platform/arch is
targets are for the compiled outputs of builds, rather than the build tool itself, i think.
so cargo being aarch is fine as long as cargo is using the relevant target/toolchain for the binary it is building
a
Huh, so I think it matters that rustup was installed on my machine in arm mode
there’s probably somewhere else that we need to specify the arch
Copy code
warning: /Users/chrisjrn/src/pants/src/rust/engine/Cargo.toml: unused manifest key: build
w
Huh, so I think it matters that rustup was installed on my machine in arm mode
it shouldn’t: that’s what i’m saying. i’m 98% sure i could use rustup/cargo compiled for my current platform to cross compile to a different platform.
a
No, I mean from the perspective of the defaults that rustup uses
anyway, that build section was ignored
👍 1
and I got an 
so
for the wrong arch
w
for example: to cross compile for the browser, you run cargo as x86, but you set a “target” of 
wasm-unknown-unknown
a
Yeah, I get that. I think it’s impacting defaults
w
looks like it needs to go in cargo “config” rather than the Cargo.toml: https://github.com/rust-lang/cargo/issues/4399 … so a 
.cargo/config.toml
at the root of the repo
a
let me look
w
@ancient-vegetable-10556: also, to take out one variable here, you can ~directly invoke cargo with 
./cargo
in the root of the repo
a
it’s re-downloading a pile of deps, which seems promising
Copy code
Error: "Must provide PYO3_CROSS_LIB_DIR environment variable when cross-compiling"
warning: build failed, waiting for other jobs to finish...
it seems to think it’s cross-compiling something
w
is the 
arch …
set such that it should not think that?
a
let me check
w
my guess is that it detects “target is x, but actual platform is y”
a
Copy code
chrisjrn@chrisjrns-MacBook-Pro pants % arch 
i386
chrisjrn@chrisjrns-MacBook-Pro pants % uname
Darwin
chrisjrn@chrisjrns-MacBook-Pro pants % uname -m
x86_64
chrisjrn@chrisjrns-MacBook-Pro pants %
w
taking a step back, you made it further than this yesterday: you had gotten to plugin loading, which is well after this step (ie pants was running successfully, including native extensions). which arch were you targeting in that case?
a
Oh, all of that was running under arm64
the problem was that there were python dependencies that weren’t available in arm64, so I switched to x86_64
w
ok… given how much further along you were there, i wonder if continuing down that path and disabling plugins if need be wouldn’t be easier
right: but only plugins… can disable those as a workaround
a
Allow me a few minutes to get back to where I was
w
for example, to disable plugins for one run you could set 
PANTS_PLUGINS='[]'
a
it doesn’t like that 🙂
oh, I see why
yeah, that all worked, I was able to successfully bootstrap pants on arm
w
ok, great. the only downside of disabling the TC plugin in Pants is that you won’t be able to use remote caching. we should figure that out in the next few weeks if we can, but if i had to guess, that is also where @hundreds-father-404 is currently.
a
My hope is that we can figure out this Rust thing at some point
w
yea, maybe. if you’re able to skip using Rosetta entirely, that’s good too.
a
Right
that’s going to require dependencies compiled for the right arch
w
so, the fact that Pants is running at all means that they are already compiled for the right arch, i think
a
Not the plugins though, right?
w
and only the plugins did not (hdrhistogram, toolchain)
correct
a
Right
h
I don't have good reception to see this whole thread. But all I do is set PY=python3.9 and PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS to 3.9. I do not use Rosetta and the arch command. Rust builds an ARM binary
I use envrc to set both env vars
Oh and I don't need to disable TC plugin anymore, now that it works with Py39. I explicitly enable BuildSense and remote cache read and write
a
OK, let’s see how that goes then
I should have clicked that I needed python 3.9 enabled
Looks like I already had Python 3.9 enabled, and the thing that was failing was the 
hdrhistogram
plugin. This, on the other hand, affects the toolchain plugin:
Copy code
Needed macosx_11_0_arm64-cp-39-cp39 compatible dependencies for:
 1: cryptography>=3.2
    Required by:
      pyOpenSSL 20.0.1
    But this pex had no 'cryptography' distributions.
 2: cryptography>=1.3.4; extra == "security"
    Required by:
      requests 2.25.1
    But this pex had no 'cryptography' distributions.
h
Oh. Run with --no-process-execution-local-cache as an experiment
a
that didn’t help
h
Hm. Is there more to the error? What command are you running?
a
I’m running 
MODE=debug  ./pants --no-process-execution-local-cache
; I have 
hdrhistogram
commented out in my config file, and there’s one more line that I missed (excluding the traceback) 
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /private/var/folders/0m/h2n902qn38b8555xq8z0w_h40000gn/T/process-executionSOzQ5z/.tmp/tmpkbur2u36.
h
What's the traceback?
a
Copy code
File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 484, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
    return runner(*args)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
    return runner(*args)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 540, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 656, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.bootstrap/pex/pex.py", line 688, in execute_pkg_resources
    return runner()
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/bin/pex.py", line 1161, in main
    seed_info = seed_cache(options, pex, verbose=options.seed == Seed.VERBOSE)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/bin/pex.py", line 1215, in seed_cache
    venv_pex = ensure_venv(pex)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/pex_bootstrapper.py", line 421, in ensure_venv
    shenbang = populate_venv_with_pex(
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/tools/commands/venv.py", line 117, in populate_venv_with_pex
    for dist in pex.resolve():
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/pex.py", line 118, in resolve
    for dist in env.resolve():
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/environment.py", line 608, in resolve
    self._resolved_dists = self.resolve_dists(all_reqs)
  File "/Users/chrisjrn/.cache/pants/named_caches/pex_root/unzipped_pexes/478cc1fa371ca40aa3e7dafee735ca438d4a243f/.deps/pex-2.1.42-py2.py3-none-any.whl/pex/environment.py", line 692, in resolve_dists
    raise ResolveError(
h
Try renaming or rm rfing ~/.cache/pants. What's happening is the cache has x86 platform in it and now you're using arm64
a
fwiw, we merged a fix last week that makes the venvs in the cache be arch-specific (https://github.com/pantsbuild/pants/pull/12192). Is there somewhere else in 
.cache
that is arch-specific?
(noting that removing the cache did indeed work)
w
there is actually… i’ll post a fix for that.
💯 1
a
thank you!
h
@witty-crayon-22786 the fix is to finish my design doc to make platform be arch aware, right?
w
uhm. mayhaps. my fix may have nothing to do with the above.
ah. so the issue above is that we’re not including the full platform/arch when invoking PEX?
h
Yeah, the resolve requirements Process stored in lmdb_store keeps OS in mind, but not arch. If we fixed that, the subsequent Pex Process should choose the correct repository pex. I think the fix is with Pants, not Pex Idk Chris's roadmap, but this would be an excellent introduction to Pants, including a little work on rust. I could write up instructions and/or video chat
w
possibly, yea. we paired on https://github.com/pantsbuild/pants/issues/12059 last week, so a good next thing maybe.
👍 1
Open in Slack
PreviousNext
Powered by