Rather foolish-feeling question; how does one hand...
# general
Rather foolish-feeling question; how does one handle tests in a separate directory from source? I may be missing something; I have (actual filenames changed):
Copy code
    python source files...
    python test files with lots of 'from mylib import...'
just has a
target to build the requirements in requirements.txt.
just has a
target with
sources=["**.py", "/***.*py"]
, and
has a
target with
. But when I do
.pants test mylib/tests
, it looks like it resolves third-party requirements (and first-party; no complaints until pytest starts), but all of the
from mylib import ...
statements get me
cannot import name XXX from mylib (unknown location)
. So there's something I'm missing in how dependencies are resolved or copied or some such. Doing
pants dependencies mylib/tests
gets me entries like
, if that helps at all. I feel like I'm missing somethang that's just stonking obvious, but I'm not clever enough to diagnose it, it seems.
your question relates to the layout of “source roots”, so good to see https://www.pantsbuild.org/docs/source-roots if you haven’t already
in this case, if your top-level
folder is at the root of the repository (there is a default source root at the root of the repository), then that might be the only source root in play
but that source root would result in your non-test code being named
if you created a source root in the top-level
folder (either by installing a pattern, or creating a marker file: see that page), then you’d get just
Aha! I had missed the source-roots business; thank you very much and apologies for the basic question. I'm fairly certain that's the issue (apologies for the delay; zoom meetings make laggards of us all). Unfortunately, as part of flailing to diagnose earlier, I reinstalled
(by removing my
directory and re-bootstrapping) and am now getting the dreaded
no module named pex.bin
error, which is most vexing; I fixed it earlier but can't for the life of me remember how. I can happily
import pex.bin
both from a local python shell and a shell if I activate the virtualenv at
so I'm utterly mystified.
👀 1
❤️ 1
Gr I don't remember how that one is fixed either, nor what causes it What's the full error?
Was 2.2.0, now trying 2.2.1rc0; Here's the trace:
Copy code
❯ ./pants repl
15:23:07.41 [INFO] Completed: Building requirements.pex
15:23:07.41 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 276, in run
    engine_result = self._run_v2(goals)
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 190, in _run_v2
    return self._maybe_run_v2_body(goals, poll=False)
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/bin/local_pants_runner.py", line 207, in _maybe_run_v2_body
    return self.graph_session.run_goal_rules(
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/init/engine_initializer.py", line 131, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 563, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/vputz/.cache/pants/setup/bootstrap-Linux-x86_64/2.2.1rc0_py38/lib/python3.8/site-packages/pants/engine/internals/scheduler.py", line 531, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building requirements.pex' failed with exit code 1.

Traceback (most recent call last):
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 487, in execute
    exit_code = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 404, in _wrap_coverage
    return runner(*args)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 435, in _wrap_profiling
    return runner(*args)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 543, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 645, in execute_entry
    return runner(entry_point)
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/pex.py", line 660, in execute_pkg_resources
    runner = entry.resolve()
  File "/home/vputz/.cache/pants/named_caches/pex_root/unzipped_pexes/1c83f6fbdf4d7132c83e9cff7eb2c16d22feee03/.bootstrap/pex/vendor/_vendored/setuptools/pkg_resources/__init__.py", line 2481, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'pex.bin'
Ugh, sorry; that's horrible.
@echoing-farmer-15630: are you running pants from inside of a separate virtualenv?
I've tried it both ways, but with a new terminal window, I still get it:
Copy code
Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish

~ on ☁️  (us-west-2) 
❯ which python

~ on ☁️  (us-west-2) 
❯ cd projects/forge

forge on  feature/vputz-try-pants [!?] via ⬢ v12.19.0 via 🐍 v3.8.6 on ☁️  (us-west-2) 
❯ ./pants repl
15:27:52.55 [INFO] Completed: Building requirements.pex
15:27:52.55 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
although... HM. Okay, a data point for this; in the process of diagnosing, I switched
interpreter_constraints = ["CPython==3.8.*"]
(which wasn't working at the time) to
, which didn't work, and now I switched it back and it restarted pantsd and is now semi-magically working again:
Copy code
forge on  feature/vputz-try-pants [!?] via ⬢ v12.19.0 via 🐍 v3.8.6 on ☁️  (us-west-2) 
❯ ./pants repl
15:29:34.34 [INFO] initializing pantsd...
15:29:34.99 [INFO] pantsd initialized.
15:29:36.51 [INFO] Completed: Building requirements.pex
Python 3.8.6 (default, Jan 27 2021, 15:42:20) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
So I apologize for the vexing bug reports; this seems peculiar to me.
👀 1
The good news is that your source roots advice was right on (and well appreciated).
very sorry for the trouble! i’m with Eric: i believe that there have been fixes for the
issue, but am having trouble locating them.
@echoing-farmer-15630: if you manage to get back in this state, i have an idea of what to try
Not to worry; apologies for not writing down my first fix (I had thought it was sourcing the pants venv and installing pex, but it's there on both counts). I tried activating another virtualenv and 2.2.1rc0 with python 3.8.6 still worked with
pants repl
. Thanks so much for your help; if I can replicated it I'll start another thread. Now to chip away at the library aliases and other requirements I haven't figured out yet!
👍 1
ok, found it: https://github.com/pantsbuild/pants/issues/11211 … but yea, if you see it again, please pass
and add some info there, or start a new thread
sounds good: thanks!
Now to chip away at the library aliases and other requirements I haven't figured out yet!
I'm not sure what part that's referring to, but we're happy to help 🙂