Rather foolish-feeling question; how does one handle tests in a separate directory from source? I may be missing something; I have (actual filenames changed):

mylib/
  requirements.txt
  BUILD
  mylib/
    python source files...
    BUILD
  tests/
    python test files with lots of 'from mylib import...'
    BUILD

mylib/BUILD

just has a

python_requirements

target to build the requirements in requirements.txt.

mylib/mylib/BUILD

just has a

python_library

target with

sources=["**.py", "/***.*py"]

, and

mylib/tests/BUILD

has a

python_tests

target with

sources=["***/*.py"]

and

dependencies=['//mylib/mylib']

. But when I do

.pants test mylib/tests

, it looks like it resolves third-party requirements (and first-party; no complaints until pytest starts), but all of the

from mylib import ...

statements get me

cannot import name XXX from mylib (unknown location)

. So there's something I'm missing in how dependencies are resolved or copied or some such. Doing

pants dependencies mylib/tests

gets me entries like

mylib/mylib/submodule/__init__.py:../../mylib

, if that helps at all. I feel like I'm missing somethang that's just stonking obvious, but I'm not clever enough to diagnose it, it seems.

your question relates to the layout of “source roots”, so good to see https://www.pantsbuild.org/docs/source-roots if you haven’t already

Aha! I had missed the source-roots business; thank you very much and apologies for the basic question. I'm fairly certain that's the issue (apologies for the delay; zoom meetings make laggards of us all). Unfortunately, as part of flailing to diagnose earlier, I reinstalled

pants

(by removing my

~/.cache/pants

directory and re-bootstrapping) and am now getting the dreaded

no module named pex.bin

error, which is most vexing; I fixed it earlier but can't for the life of me remember how. I can happily

import pex.bin

both from a local python shell and a shell if I activate the virtualenv at

~/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38

so I'm utterly mystified.

Gr I don't remember how that one is fixed either, nor what causes it What's the full error?

Was 2.2.0, now trying 2.2.1rc0; Here's the trace:

❯ ./pants repl
/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38
15:23:07.41 [INFO] Completed: Building requirements.pex
15:23:07.41 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 276, in run
    engine_result = self._run_v2(goals)
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 190, in _run_v2
    return self._maybe_run_v2_body(goals, poll=False)
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 207, in _maybe_run_v2_body
    return self.graph_session.run_goal_rules(
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/init/engine_initializer.py", line 131, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 563, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 531, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building requirements.pex' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 487, in execute
    exit_code = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 404, in _wrap_coverage
    return runner(*args)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 435, in _wrap_profiling
    return runner(*args)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 543, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 645, in execute_entry
    return runner(entry_point)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 660, in execute_pkg_resources
    runner = entry.resolve()
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/vendor/_vendored/setuptools/pkg_resources/__init__.py", line 2481, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'pex.bin'

@echoing-farmer-15630: are you running pants from inside of a separate virtualenv?

I've tried it both ways, but with a new terminal window, I still get it:

Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish

~ on ☁️  (us-west-2) 
❯ which python
/usr/bin/python

~ on ☁️  (us-west-2) 
❯ cd projects/forge

forge on  feature/vputz-try-pants [!?] via ⬢ v12.19.0 via 🐍 v3.8.6 on ☁️  (us-west-2) 
❯ ./pants repl
/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38
15:27:52.55 [INFO] Completed: Building requirements.pex
15:27:52.55 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
...

although... HM. Okay, a data point for this; in the process of diagnosing, I switched

interpreter_constraints = ["CPython==3.8.*"]

(which wasn't working at the time) to

3.9.*

, which didn't work, and now I switched it back and it restarted pantsd and is now semi-magically working again:

forge on  feature/vputz-try-pants [!?] via ⬢ v12.19.0 via 🐍 v3.8.6 on ☁️  (us-west-2) 
❯ ./pants repl
/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38
15:29:34.34 [INFO] initializing pantsd...
15:29:34.99 [INFO] pantsd initialized.
15:29:36.51 [INFO] Completed: Building requirements.pex
Python 3.8.6 (default, Jan 27 2021, 15:42:20) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>

So I apologize for the vexing bug reports; this seems peculiar to me.

very sorry for the trouble! i’m with Eric: i believe that there have been fixes for the

pex.bin

issue, but am having trouble locating them.

Not to worry; apologies for not writing down my first fix (I had thought it was sourcing the pants venv and installing pex, but it's there on both counts). I tried activating another virtualenv and 2.2.1rc0 with python 3.8.6 still worked with

pants repl

. Thanks so much for your help; if I can replicated it I'll start another thread. Now to chip away at the library aliases and other requirements I haven't figured out yet!

ok, found it: https://github.com/pantsbuild/pants/issues/11211 … but yea, if you see it again, please pass

--pex-verbosity=9

and add some info there, or start a new thread

Now to chip away at the library aliases and other requirements I haven't figured out yet!

I'm not sure what part that's referring to, but we're happy to help 🙂