Hello! I've a monorepo laid out as such: mondo - ...
# general
p
Hello! I've a monorepo laid out as such: mondo - root of the monorepo where there is a pyenv dedicated to the 'top' of this repo and pants installed via bootstrap (mondo-3.8.10 via mondo/.python-project) mondo/src/python/project1 - this project as it's own pyenv (project1-3.8.10 via mondo/src/python/project1/.python-project) mondo/tests/python/project1 - this dir uses the same pyenv as src/python/project1 via mondo/tests/python/project1/.python-project with name project1-3.8.10) pyenv mondo-3.8.10 is a minimal pyenv containing only modules to support pants. pyenv project1-3.8.10 contains all of the required modules for coding project1. In my case the AWS Boto3 modules. pyenv activates/deactivates correctly in the terminal when cd'ing into/out of the venv - so, the local dir .python-version is, I'm presuming, accurate for pants purposes. Most every other goal appears to use the project1 pyenv for the stated goal. e.g. ~/mondo:*./pants package src/python/project1:dist* or ./pants typecheck src/python/project1 all complete successfully. However, when I attempt the test goal the tests fail with modules not found. e.g. ~/mondo:*./pants test tests/python/project1:tests* fails. Tried: ā€¢ *pants.to*ml interpreter_search_paths = ["<PATH>", "<PYENV_LOCAL>", "<PYENV>"] and exporti PATH=/home/x/.pyenv/versions/project1-3.8.10/bin:$PATH ā€¢ *pants.to*ml interpreter_search_paths = ["*/home/x/.pyenv/versions/project1-3.8.10/bin"*] ā€¢ Copy tests/python/project1 to src/python/project1/tests and setting BUILD in src/python/project1 python_tests( name="tests", sources = [ "tests/*_test.py", "tests/*_tests.py", "tests/*_test.pyi", "tests/*_tests.pyi", ], ) ~/mondo:*./pants src/python/project1:tests* tests fail with the same module not found errors. I've confirmed the project1-3.8.10 virtualenv has the modules (mondo-3.8.10 does not have them nor should ever need them). Via a terminal, if I cd to ~/mondo/tests/python/project1 the pyenv project1-3.8.10 is activated. Then *pytest _test.py completes successfully for all tests. I've searched Pants documentation, https://github.com/pantsbuild/pants/issues and this Slack account but I've not found anything illuminating. When, running a test goal - is it a correct assumption that the target pyenv should be used for the tests? Are there any flagrant missteps I've done that would prevent an independent projects tests not to execute under the projects venv? Has anyone else had similar issues? Is the next step opening an issue in GitHub? If so, are there any particular artifacts or information to include beyond what I've described? Thank you!
h
Hello, welcome @polite-angle-19513! I'm glad you reached out, this seems frustrating and I'm happy to try to help šŸ™‚
./pants test
should be respecting
[python-setup].interpreter_search_paths
still, like everything else. I'm surprised it is not To debug this, it would be helpful to do a run w/
interpreter_search_paths
set to
"/home/x/.pyenv/versions/project1-3.8.10/bin"
, and also
-ldebug
and
--pex-verbosity=3
. It's going to be a ton of output, so if you could please put it in a Gist or Pastebin etc, that'd be great
Also, what is
[python-setup].interpreter_constraints
set to? And are you using the
interpreter_constraints
field anywhere?
p
Thank you for the prompt reply! I'll work on the debug output and update thread once I've collected it. As for the [python-setup], this are the arguments defined in the pants.toml: [python-setup] interpreter_constraints = [">=3.7"] interpreter_search_paths = ["<PATH>", "<PYENV_LOCAL>", "<PYENV>"]
h
Cool, thanks. As an aside, does your project need to work with multiple interpreter versions? If you're able to pin to something like
==3.9.*
, it's often a good idea. Reduces the risk of "Works On My Machine" like errors
p
I've set interpreter_constraints = ["==3.8.*"] run the goal and completed collecting the debug for ./pants test tests/python/hank_ai_aws:tests https://gist.github.com/james-crocker/eb56f2ca4065b3c7783107569b6d5d1c And, indeed quite a bit of info. Have a great weekend and I look forward to any insights. Thank you.
Oh, and I also ran the last goal with: interpreter_search_paths = ["/home/x/.pyenv/versions/3.8.10/envs/hank_ai_aws-3.8.10/bin"]
Well, Eric - that helped. I've got the tests working correctly now. I'd missed on the terminal runs where it was [WARN] for dependency imports that could not be inferred. I also glossed over it as it was a 'warn' level. Once I added the 3rdparty/python dependencies to the BUILD files, the warnings vanished and the tests completed successfully. (I'm going to remove the Gist). Again, have a great weekend and thank you for all the help!
h
Great to hear! If you don't mind tho, it'd be helpful to keep the gist. I'm confused why it wasn't doing what you expected before
Let us know also if there's anything else we can help with, or if you have feedback along the way! We're particularly focusing right now on making it easier to adopt Pants incrementally, including if you already have a large repo
p
Oops, I'd already deleted. Setback to fail state and re-ran. Here's the Gist. If you'd be so kind when through to let me know I can delete it I'd appreciate it. https://gist.github.com/james-crocker/0096f1ff51fa9ab32ef7ec82ed7b0ab6
šŸ‘ 1
h
Is it okay if I save the file locally? No worries if you don't want that data persisted
p
Okay local.
h
Thanks! downloaded
h
Hi @polite-angle-19513, just wanted to follow up and see how your Pants adoption is going, and if we can help in any way?
p
Afternoon - doing much better, things are working nicely with pants lint, typecheck and package - especially with cross project dependencies. It just took some time to get my head around the proper BUILD syntax. But, working relatively well thank you. Now just tackling the IDE tooling tweaks for pylint, mypy, pylance, etc. to work together. One note on the pantsd daemon. Have found that if there is a significant code refactor (dir moves/renames, etc) that I need to kill the daemon and clear out the ~/.cache/pants, .pants.d and .pids to get a proper run of things. But, that is a seldom thing thankfully.