Hi, team I am getting following error when running pants goals

Bootstrapping Pants using /Users/anoopmishra/miniconda3/bin/python3.9
Creating the virtualenv PEX.
Downloading the Pex PEX.
SHA256 fingerprint of <https://github.com/pantsbuild/pex/releases/download/v2.1.62/pex> verified.
Installing pantsbuild.pants==2.13.0 into a virtual environment at /Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39
New virtual environment successfully created at /Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39.
Traceback (most recent call last):
  File "/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/bin/pants", line 5, in <module>
    from pants.bin.pants_loader import main
  File "/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 19, in <module>
    from pants.bin.pants_runner import PantsRunner
  File "/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/pants/bin/pants_runner.py", line 11, in <module>
    from pants.base.exception_sink import ExceptionSink
  File "/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/pants/base/exception_sink.py", line 16, in <module>
    import psutil
  File "/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/psutil/__init__.py", line 123, in <module>
    from . import _psosx as _psplatform
  File "/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/psutil/_psosx.py", line 14, in <module>
    from . import _psutil_osx as cext
ImportError: dlopen(/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so, 0x0002): tried: '/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/_psutil_osx.cpython-39-darwin.so' (no such file), '/usr/lib/_psutil_osx.cpython-39-darwin.so' (no such file), '/Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/pants.iUu4Nv/install/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/_psutil_osx.cpython-39-darwin.so' (no such file), '/usr/lib/_psutil_osx.cpython-39-darwin.so' (no such file)

Env: • using miniconda3 python • Apple M1 chip • Mac 12.0.1

Hi! Sorry you’re having issues. I don’t know if Pants runs on miniconda, or at least it’s not how it’s usually run… Can you try with a vanilla CPython (e.g., one installed by pyenv)?

Not sure that’s related to this error, but want to rule it out

Oh, also, what does

file /Users/anoopmishra/.cache/pants/setup/bootstrap-Darwin-arm64/2.13.0_py39/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so

show?

For context: psutil doesn’t publish macos arm64 wheels, so when the venv is created during setup, the sdist will be downloaded and built.

Looks like that is being built as an x86_64 binary, even on your arm64 system, which can happen if something is running under rosetta

So my guess is that your miniconda3 interpreter is an x86_64 binary, running under rosetta?

Which you can check with

file path/to/bin/python

If as a newcomer you'd gotten a brief weekly "Pants Tips" (consensual opt-in) newsletter a handful of times, would this have been helpful? I periodically get newsletters like there when I sign up for a new SaaS or install an especially complicated app. Usually there's one email reminding me of the first few setup/config steps, one a week or two later that assumes I've gotten started but could use a reminder of how to seek help if needed, one later that assumes I'm a little farther along in exploring and wondering what else it can do, maybe just one more after that, then it typically stops. I find a couple of widely-spaced prompts handy when the new software isn't something I can explore thoroughly in one session, as long as the emails are very few and clearly unselfish in motivation (no upsell). Do others feel similarly? I'm wondering if it would be helpful enough to justify the effort involved in setting up an automation of this nature.

I think I'm having an issue that goes a bit beyond the help of

module_mappings

though. I'm getting errors trying to resolve packages like yaml click, and six even with a module mapping. Not to mention the default module mapping should already cover the YAML case. I'll investigate it later but I've been seeing this for a while.

Weird, all those should be mapped correctly (pyyaml is in the default mapping, and six and click have the same name as their top-level package, so they don’t need explicit mapping)

Is this error happening during lockfile generation?

Here is a fun possible bug relating to using version controlled direct references hosted on private repositories with SSH access:

surprisingly, when I ran

file

on the

cpython-39-darwin.so

file it's x86_64 but i never see this issue on my m1 under arm64 mode

Hi, I am using pants to build and run some go tests. the test would invoke certain go commands, but I encountered errors "err: go command required, not found: exec: "go": executable file not found in $PATH". does pants have go command on PATH?

How hard would it be to support absolute paths for resource targets’ sources? The issue originates from the following dilemma: our dev environment assumes access to a globally replicated network file system, where lots of resources and binaries are installed, but bringing those resources into an application at runtime is a practice we’d prefer to avoid (having it documented in a BUILD file is much more maintainable and easier to track dependencies from) and is not possible anyway when the dist target is Docker or some other environment that does not have such access. On the other hand, bringing those resources under the project requires us to manually keep them up to date ourselves, not to mention potentially resulting in lots of things you’d rather not have in source control. From our perspective, allowing Pants to bundle resources and data into binaries and distros from network paths is a good solution that balances these issues.

We are unable to generate a pex lockfile for our repository 😞

hey folks, is there a way to debug dependency inference for python? I've got an import statement in a test that isn't generating the corresponding dependency

There's a warning that happens when running

pants check

with MyPy. Lockfile generation is fine. I'll see if I can reconstruct a simple repro

Ah so you may need to add type stubs for those libraries to your requirements.txt (or specify them in `[mypy].extra_type_stubs`: https://www.pantsbuild.org/docs/reference-mypy#extra_type_stubs

So you don’t see it if you run

./pants dependencies path/to/test.py

?

One thing to check is whether the dependency’s source root is correctly listed in

./pants roots

Ohhh, it's almost certainly that the roots are messed up. I had a root at "/a/b/x" and "/c/d/x", but mypy was complaining that the module "x" appeared multiple times (fair). So I modified the modules to use the full path "from a.b.x import ...", But didn't remove the roots. I'll give that a shot tomorrow. More generally, is the a way to enable debug logging for dependency inference? Ideally something like "found import statement ... , Resolves to target ...". My usual

-ldebug

only gave a single line for the process.

Alas I don’t think so, but that would be good to add to

-ldebug

Oh, have you tried

-ltrace

?

The dependency inference script is embedded in pants and dependency-less. Sometimes to debug it I just run it directly. Take note of the comment at the top for how to run it (it expects from env vars) https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/dependency_inference/scripts/dependency_parser_py

yep, that was it. removing the (nested) extraneous roots.

-ldebug

and

-ltrace

didn't add anything, just progress notifications. Manually running dependency_parser_py does resolve the import correctly, so I think it's afterward that it gets dropped.

Oh no I did add the types- for these packages. The problem isn't specific to MyPy. I think it warns during things like tests too. I'm not sure if the problem actually manifests as a runtime error though

I think not because tests generally pass or at least don't fail on transitive import failures