https://pantsbuild.org/ logo
Join Slack
Powered by
I am trying to get my Django tests running with pa...
# general
t
I am trying to get my Django tests running with pants, but am struggeling: https://github.com/pantsbuild/example-django/issues/24 Can anyone help?
✅ 1
👀 1
c
Hi Chris, A couple of questions.. • Does 
./pants test ::
in the example-django project work for you? (to eliminate if this is an issue related to your environment you run in) • Is your setup and error exactly the same as the issue you linked to above? • Are you in a position where you can push an example repro that we can play along with?
t
Hi Andreas, • No 
./pants test ::
does not do the trick • Yes it is • I have shared my code here. Note that I am running it within the VSCode devcontainer I would really appreciate some help here. Thanks in advance! 🙏
c
OK, I get an error on your example repo too (when not using the devcontainer, haven’t tried using it)
Ah, it seems that django requires the 
settings.py
file from 
mysite
but there’s no explicit dependency for that, so Pants doesn’t infer it, so when you run the tests in a sandbox, mysite is not present there. This is the crucial part:
Copy code
diff --git a/django/polls/BUILD b/django/polls/BUILD
index 0eea8b1..72520e3 100644
--- a/django/polls/BUILD
+++ b/django/polls/BUILD
@@ -2,4 +2,5 @@ python_sources()
 
 python_tests(
     name="tests",
+    dependencies=["django/mysite/settings.py"],
 )
I also commented out 
pythonpath
and 
python_files
in 
pytest.ini
but I don’t think that is relevant here.
And with that I get:
Copy code
$ ./pants test django/polls::
13:06:25.00 [ERROR] Completed: Run Pytest - django/polls/tests.py:tests failed (exit code 1).
============================= test session starts ==============================
platform darwin -- Python 3.9.13, pytest-7.1.2, pluggy-1.0.0
django: settings: mysite.settings (from ini)
rootdir: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionC6hdoG, configfile: pytest.ini
plugins: django-4.5.2, cov-3.0.0
collected 2 items

django/polls/tests.py F.                                                 [100%]

=================================== FAILURES ===================================
_________________ YourTestClass.test_something_that_will_fail __________________

self = <polls.tests.YourTestClass testMethod=test_something_that_will_fail>

    def test_something_that_will_fail(self):
>       self.assertTrue(False)
E       AssertionError: False is not true

django/polls/tests.py:16: AssertionError
- generated xml file: /private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionC6hdoG/django.polls.tests.py.tests.xml -
=========================== short test summary info ============================
FAILED django/polls/tests.py::YourTestClass::test_something_that_will_fail - ...
========================= 1 failed, 1 passed in 0.72s ==========================
> Does 
./pants test ::
in the example-django project work for you?
• No 
./pants test ::
does not do the trick
To be clear, did you try this on https://github.com/pantsbuild/example-django ? If that does not work, something is likely broken, either in your environment, or the repo is not configured properly for your system.. it works on my MacBook Pro at least.
t
Ah okay, I see. This makes totally sense. If the entrypoint would be manage.py it should be infered as there is a string import available. Is there a way to add this dependency to all my django test targets automatically?
No I did not try to run the tests on https://github.com/pantsbuild/example-django, I was only referring to my specific setup. So I guess it would work in my environment as well.
👌 1
c
Is there a way to add this dependency to all my django test targets automatically?
Currently, not directly. You can use macros to use a custom version of the 
python_tests
target which adds this dependency that way. Another option is around the corner, once the new 
__defaults__
feature has landed: https://github.com/pantsbuild/pants/pull/15836
t
Looks good, will keep an eye on that PR and apply it once available. Thank you very much for your help!
👍 2
🙏 2
I have seen that this PR has been merged and want to give it a try. Therefore, I have installed the latest SHA of the main branch and added this to my 
./django/BUILD
:
Copy code
__defaults__({
      (python_tests): dict(dependencies=["django/django_core/settings.py"])
    })
Running 
./pants tailor
results in:
Copy code
07:37:18.97 [ERROR] 1 Exception encountered:

  MappingError: Failed to parse ./django/BUILD:
Name '__defaults__' is not defined.

If you expect to see more symbols activated in the below list, refer to <https://www.pantsbuild.org/v2.11/docs/enabling-backends> for all available backends to activate.
Am I missing something or is it not ready to be tried?
c
Thanks for trying this out! I haven’t actually thought of trying this with the 
tailor
goal, so may very well have found a real issue here. Let me have a look at it.
OK, I’ve tried with this sha: 
PANTS_SHA=6c15bcd23c831541012414914d1b488eb9b5a2c0
and it works better then. Although there is another issue surfaced than this is a little nuanced (will get into that in a following comment)
Copy code
$ PANTS_SHA=6c15bcd23c831541012414914d1b488eb9b5a2c0 ./pants tailor django
09:48:42.24 [ERROR] 1 Exception encountered:

  MappingError: Failed to parse ./django/BUILD:
Unrecognized field `dependencies` for target python_tests. Valid fields are: description, overrides, sources, tags.
Here, 
dependencies
is not technically a field of 
python_tests
(the target generator) but a field for the generated targets of type 
python_test
(in the Pants internals, this is known as a 
moved field
) What this means is, that to set default dependencies, you need to address the 
python_test
target, rather than the target generator.
I think however that we should support providing defaults also for moved fields. We’ll be looking at fixing this.
i.e. this works:
Copy code
__defaults__({
      (python_test): dict(dependencies=["django/django_core/settings.py"])
    })
Uhm… or at least doesn’t blow up… (looking at 
./pants peek
to verify it actually picks up the dependency)
Doh. Nope, sorry. The defaults doesn’t bite on generated targets. So for moved fields, it is a no-go. Will work on a fix asap.
Thanks again for flagging!
t
Ah okay, than I slightly misunderstood the actual usage. It is a bit weird as I cannot reproduce the error above `Unrecognized field 
dependencies
for target python_tests` with that SHA. I have pushed the changes to the example repo https://github.com/ptrhck/example-django Happy to try things 😊
c
Great. I’ll post the PR here so you can see when the fix gets merged, and I’ll have another look at your example repo with it then.
👍 1
I may not get it out today, though (it’s a public holiday in Sweden today) - just FYI..
(oh, it was less involved than expected 😄 )
t
Great, thanks! 🙂 How can I try this for my example repo?
c
Before it’s landed on 
main
, you can try this out if you check out the pants repo next to your django repo (as siblings), and then invoke a 
./pants_from_sources
script in your django repo, after copying it from https://github.com/pantsbuild/example-python/blob/main/pants_from_sources
t
Ah okay great, thank you! I have a question that is related to this: I did 
PANTS_SHA=6c15bcd23c831541012414914d1b488eb9b5a2c0 ./pants --version
to use the main branch. This PR https://github.com/pantsbuild/pants/pull/15799 has been merged into main, but running 
./pants --debug-adapter test ::
results in 
Unknown flag --debug-adapter on global scope
. Am I missing something to try features of the main branch?
c
OK, so this is actually where how pants scopes options a bit unusual shines through. The options is actually 
--test-debug-adapter
but when you provide options after a goal, you can omit the goals name (the scope) of the option. So these are all the same:
Copy code
./pants --test-debug-adapter test ...
./pants test --debug-adapter ...
./pants test --test-debug-adapter ...
Copy code
--[no-]test-debug-adapter
  PANTS_TEST_DEBUG_ADAPTER
  debug_adapter
      default: False
      current value: False
      Run tests sequentially in an interactive process, using a Debug Adapter (<https://microsoft.github.io/debug-adapter-protocol/>) for the language if supported.
      
      The interactive process used will be immediately blocked waiting for a client before continuing.
      
      This option implies --debug.
🙌 1
t
In the https://github.com/ptrhck/example-django repo, 
./pants run django:manage -- runserver
is not working as it fails with:
Copy code
File "/workspaces/example-docker-django/.pants.d/tmp2njknb9j/django/manage.py", line 10, in main
    from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspaces/example-docker-django/.pants.d/tmp2njknb9j/django/manage.py", line 21, in <module>
    main()
  File "/workspaces/example-docker-django/.pants.d/tmp2njknb9j/django/manage.py", line 12, in main
    raise ImportError(
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
Running `./pants run django:manage -- migrate`works though. Any idea?
Still struggeling and would very much appreciate help. Any idea @curved-television-6568 ?
👀 1
c
The issue seems to be related to how django manage works… it runs your script twice in case of 
runserver
, but not with the a preserved `sys.path`:
Copy code
$ PANTS_SHA=7382ad47b42618b155d8c3a0ea63b3827bdca574 ./pants run django:manage -- runserver

sys path:
 - /Users/aadt/.cache/pants/named_caches/pex_root/unzipped_pexes/6c5bb92cd3381912cfefb4f4ff1d424ef418e304
 - /Users/aadt/.cache/pants/named_caches/pex_root/unzipped_pexes/271fc089333a4630f72007fa0b476eab7e03e401
 - /usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python39.zip
 - /usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9
 - /usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
 - /Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django
 - /Users/aadt/.cache/pants/named_caches/pex_root/installed_wheels/b896ca61edc079eb6bbaa15cf6071eb69d6aac08cce5211583cfb41515644fdf/Django-3.2.13-py3-none-any.whl
 - /Users/aadt/.cache/pants/named_caches/pex_root/installed_wheels/1d2880b792ae8757289136f1db2b7b99100ce959b2aa57fd69dab783d05afac4/asgiref-3.5.2-py3-none-any.whl
 - /Users/aadt/.cache/pants/named_caches/pex_root/installed_wheels/e68985985296d9a66a881eb3193b0906246245294a881e7c8afe623866ac6a5c/pytz-2022.1-py2.py3-none-any.whl
 - /Users/aadt/.cache/pants/named_caches/pex_root/installed_wheels/48719e356bb8b42991bdbb1e8b83223757b93789c00910a616a071910ca4a64d/sqlparse-0.4.2-py3-none-any.whl
 - /Users/aadt/.cache/pants/named_caches/pex_root/unzipped_pexes/271fc089333a4630f72007fa0b476eab7e03e401/.bootstrap

enter main for: ['/Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django/manage.py', 'runserver']...

EXEC MANAGE: ['/Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django/manage.py', 'runserver']...


sys path:
 - /Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django
 - /usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python39.zip
 - /usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9
 - /usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
 - /Users/aadt/Library/Python/3.9/lib/python/site-packages
 - /usr/local/lib/python3.9/site-packages

enter main for: ['/Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django/manage.py', 'runserver']...
Traceback (most recent call last):
  File "/Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django/manage.py", line 11, in main
    from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django/manage.py", line 25, in <module>
    main()
  File "/Users/aadt/src/github/ptrhck/example-django/.pants.d/tmpv46zyalz/django/manage.py", line 13, in main
    raise ImportError(
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
which seems like it would loop, any way…. so something feels a bit off here…
I was using this diff:
Copy code
$ git diff
diff --git a/django/manage.py b/django/manage.py
index 3d1e46c..7d5faba 100755
--- a/django/manage.py
+++ b/django/manage.py
@@ -1,7 +1,8 @@
-#!/usr/bin/env python
 """Django's command-line utility for administrative tasks."""
 import os
 import sys
+sys_path = "\n - ".join(sys.path)
+print(f"\nsys path:\n - {sys_path}\n")
 
 def main():
     """Run administrative tasks."""
@@ -14,8 +15,11 @@ def main():
             "available on your PYTHONPATH environment variable? Did you "
             "forget to activate a virtual environment?"
         ) from exc
+
+    print(f"\nEXEC MANAGE: {sys.argv}...\n")
     execute_from_command_line(sys.argv)
 
 
 if __name__ == '__main__':
+    print(f"enter main for: {sys.argv}...")
     main()
indeed migrate works, as it does not call into the manage.py script a second time
so feels like a django/config issue with how runserver is supposed to work.
t
Thanks for looking into it, I posted this here again https://github.com/pantsbuild/example-django/issues/24
6 Views
Open in Slack
PreviousNext
Powered by