https://pantsbuild.org/ logo
h

hundreds-father-404

03/09/2021, 8:36 PM
Two new Pants releases: + 2.3.0rc2 https://pypi.org/project/pantsbuild.pants/2.3.0rc2 + 2.4.0.dev1 https://pypi.org/project/pantsbuild.pants/2.4.0.dev1 Feedback particularly appreciated on 2.3.0! It's overdue for a stable release
c

clean-city-64472

03/09/2021, 11:56 PM
2.3.0rc2 passes some test suites and then just hangs on a couple of them seemingly forever...
👀 1
h

hundreds-father-404

03/09/2021, 11:58 PM
Hm, in CI, right?
c

clean-city-64472

03/10/2021, 12:05 AM
locally - i switched it back to 2.2.0 and everything works
Copy code
vagrant@ubuntu-focal:/vagrant$ ./pants test tests/python/clofo_test::
23:53:00.34 [INFO] initializing pantsd...
23:53:00.59 [INFO] pantsd initialized.
23:53:03.25 [WARN] /home/vagrant/.cache/pants/setup/bootstrap-Linux-x86_64/2.3.0rc2_py38/lib/python3.8/site-packages/pants/util/memo.py:123: DeprecationWarning: DEPRECATED: defaulting to the legacy pip resolver will be removed in version 2.4.0.dev0.
  In Pants 2.4.0, Pants will default to using pip's new resolver. The legacy resolver will be removed in Pants 2.5.

Set `[python-setup].resolver_version = 'pip-2020-resolver` to prepare for the default changing. Refer to <https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020> for more information on the new resolver.
  result = func(*args, **kwargs)

23:53:03.91 [INFO] Completed: test - tests/python/clofo_test/qpop/test_metrics.py:qpop_tests succeeded.
23:53:03.92 [INFO] Completed: test - tests/python/clofo_test/qpop/test_queue.py:qpop_tests succeeded.
23:53:03.92 [INFO] Completed: test - tests/python/clofo_test/qpop/test_scheduler.py:qpop_tests succeeded.
23:53:05.49 [INFO] Completed: test - tests/python/clofo_test/compare/test_cleanup.py:compare_tests succeeded.
23:53:05.51 [INFO] Completed: test - tests/python/clofo_test/compare/test_job.py:compare_tests succeeded.
23:53:06.16 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_tasks.py:test_tasks succeeded.
23:53:06.17 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_serializers.py:test_serializers succeeded.
23:53:06.18 [INFO] Completed: test - tests/python/clofo_test/predicter/lib/predicter/test_resnet.py succeeded.
23:53:06.18 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_auth.py:test_auth succeeded.
23:53:06.18 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_middleware.py:test_middleware succeeded.
23:53:06.18 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_utils.py:test_utils succeeded.
23:53:08.49 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_ember_views.py:test_ember_views succeeded.
23:53:08.85 [INFO] Completed: test - tests/python/clofo_test/auth/users/test_models.py:test_models succeeded.
23:54:30.76 [INFO] Long running tasks:
  81.91s 	Run Pytest for tests/python/clofo_test/auth/users/test_views.py:test_views
  82.28s 	Run Pytest for tests/python/clofo_test/auth/users/saml/test_views.py
23:55:00.80 [INFO] Long running tasks:
  111.94s 	Run Pytest for tests/python/clofo_test/auth/users/test_views.py:test_views
  112.31s 	Run Pytest for tests/python/clofo_test/auth/users/saml/test_views.py
23:55:30.82 [INFO] Long running tasks:
  141.97s 	Run Pytest for tests/python/clofo_test/auth/users/test_views.py:test_views
  142.33s 	Run Pytest for tests/python/clofo_test/auth/users/saml/test_views.py
23:56:00.82 [INFO] Long running tasks:
  171.97s 	Run Pytest for tests/python/clofo_test/auth/users/test_views.py:test_views
  172.34s 	Run Pytest for tests/python/clofo_test/auth/users/saml/test_views.py
23:56:30.91 [INFO] Long running tasks:
  202.06s 	Run Pytest for tests/python/clofo_test/auth/users/test_views.py:test_views
  202.42s 	Run Pytest for tests/python/clofo_test/auth/users/saml/test_views.py
Is there a way to stream output while the tests are running?
w

witty-crayon-22786

03/10/2021, 12:10 AM
only in
--debug
mode, unfortunately.
…but that is worth trying.
@clean-city-64472: how long do those usually take to run?
./pants test --debug tests/python/clofo_test/auth/users/test_views.py:test_views
would run one test in the foreground
c

clean-city-64472

03/10/2021, 12:14 AM
i left it and it got over 3,000 seconds, switched back to 2.2 and it ran teh whole suite within a minute
just double checking that caching wasn't in play...
Yeah it's definitely related to the upgrade. It seems to hang on a particular test - but I don't get why it would be that test - there is nothing unique about it.
w

witty-crayon-22786

03/10/2021, 12:23 AM
can you try running it the foreground with the
--debug
flag? see above
and do you see CPU usage while it is running?
c

clean-city-64472

03/10/2021, 12:24 AM
it's very strange - it hangs on a specific line in this test - seems very application code specific - but so directly tied to the upgrade
wondering if something changed in how requirements are resolved and now some dependency is different and causing prolbems
have to go for now but will look into it a bit more later on
w

witty-crayon-22786

03/10/2021, 12:25 AM
ok, sorry for the trouble. we’ll be around.
c

clean-city-64472

03/10/2021, 3:05 AM
TESTING = "test" in sys.argv[0]
this no longer evaluates
True
when running tests in 2.3.0rc2
Is there a better way to infer from within the python environment whether it is in a test run?
w

witty-crayon-22786

03/10/2021, 3:21 AM
Hm! What does it say that
sys.argv
is? This is a forked process, so it should have lots of references to
pytest
in argv
c

clean-city-64472

03/10/2021, 3:22 AM
Copy code
INTERNALERROR>     raise Exception(sys.argv)
INTERNALERROR> Exception: ['/home/vagrant/.cache/pants/named_caches/pex_root/venvs/0faf5898821e94278b563265573c7be936bbe670/bbbad62975f48f0ab151c1c0805c4546486d65d4/pex', 'tests/python/clofo_test/auth/users/test_auth.py']
w

witty-crayon-22786

03/10/2021, 3:22 AM
But yea, the performance optimizations in 2.3.x overhauled how processes are invoked, so args will have moved around
c

clean-city-64472

03/10/2021, 3:22 AM
I changed it up to use an enviornment variable that is hardcoded to true through the pants.toml text config
👍 1
w

witty-crayon-22786

03/10/2021, 3:25 AM
That's a good idea, yea.