https://pantsbuild.org/ logo
Join Slack
Powered by
I am facing some strange behaviour when running dj...
# general
t
I am facing some strange behaviour when running django commands with pants and I am a bit lost where to look at: 1. The server just keeps restarting when opening its URL in the browser. Note: django hot code reloading is disabled, but pants reloading is enabled. Here is a log:
Copy code
pants run django/manage.py -- runserver 0.0.0.0:8000 --noreload 

12:46:22.72 [INFO] Completed: Building 48 requirements for lib.pex from the lockfiles/default.lock resolve: Cerberus==1.3.2, Fiona==1.8.20, Pillow==9.0.1, Shapely==1.8.5, aiobotocore[boto3]==2.5.0, aiohttp==3.8.3, asgiref==3.6.0, ... (1097 characters truncated)
/nefino_li/libs/common/nefino_common/geoFunctions.py:92: ShapelyDeprecationWarning: Setting custom attributes on geometry objects is deprecated, and will raise an AttributeError in Shapely 2.0
  pnt.srid = srid
Performing system checks...

/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/langchain/text_splitter.py:916: DeprecationWarning: invalid escape sequence \c
  "\n\\\chapter{",
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/langchain/text_splitter.py:917: DeprecationWarning: invalid escape sequence \s
  "\n\\\section{",
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/langchain/text_splitter.py:903: DeprecationWarning: invalid escape sequence \*
  "\n\*\*\*+\n",
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/numexpr/necompiler.py:264: DeprecationWarning: invalid escape sequence \;
  _forbidden_re = re.compile('[\;[\:]|__|\.[abcdefghjklmnopqstuvwxyzA-Z_]')
numexpr.utils INFO 2023-09-14 12:46:28,035 utils Note: NumExpr detected 16 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
numexpr.utils INFO 2023-09-14 12:46:28,035 utils NumExpr defaulting to 8 threads.
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/langchain/document_loaders/acreom.py:54: DeprecationWarning: invalid escape sequence \[
  content = re.sub("\[\[.*?\]\]", "", content)  # rm doclinks
System check identified no issues (0 silenced).
September 14, 2023 - 12:46:30
Django version 3.2, using settings 'django_core.settings'
Starting development server at <http://0.0.0.0:8000/>
Quit the server with CONTROL-C.
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/xarray/core/pycompat.py:24: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  dask_version = LooseVersion("0.0.0")
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/setuptools/_distutils/version.py:345: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  other = LooseVersion(other)
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/xarray/core/npcompat.py:131: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if LooseVersion(np.__version__) >= "1.20.0":
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/xarray/core/pdcompat.py:45: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if LooseVersion(pd.__version__) < "0.25.0":
<frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 232 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 232 from PyObject
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/statsmodels/compat/pandas.py:12: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  version = LooseVersion(pd.__version__)
<frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 232 from PyObject
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/statsmodels/tools/numdiff.py:51: DeprecationWarning: `np.MachAr` is deprecated (NumPy 1.22).
  EPS = np.MachAr().eps
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/statsmodels/tsa/base/tsa_model.py:7: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  from pandas import (to_datetime, Int64Index, DatetimeIndex, Period,
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/statsmodels/tsa/base/tsa_model.py:7: FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  from pandas import (to_datetime, Int64Index, DatetimeIndex, Period,
<frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 232 from PyObject
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/statsmodels/compat/scipy.py:6: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  SP_VERSION = LooseVersion(scipy.__version__)
/tmp/named_caches/pex_root/venvs/s/51b8e217/venv/lib/python3.8/site-packages/statsmodels/compat/numpy.py:46: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  NP_LT_114 = LooseVersion(np.__version__) < LooseVersion('1.14')
/nefino_li/libs/common/nefino_common/geoFunctions.py:92: ShapelyDeprecationWarning: Setting custom attributes on geometry objects is deprecated, and will raise an AttributeError in Shapely 2.0
  pnt.srid = srid
Performing system checks...

numexpr.utils INFO 2023-09-14 12:46:39,694 utils Note: NumExpr detected 16 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
numexpr.utils INFO 2023-09-14 12:46:39,694 utils NumExpr defaulting to 8 threads.
System check identified no issues (0 silenced).
September 14, 2023 - 12:46:40
Django version 3.2, using settings 'django_core.settings'
Starting development server at <http://0.0.0.0:8000/>
Quit the server with CONTROL-C.
2. When I run the django migrations with pants, out of the sudden KeyboardInterrupts appear. I am not hitting CTRL+C or anything:
Copy code
vscode ➜ /nefino_li (develop) $ pants run django/manage.py -- migrate
12:29:49.99 [INFO] Completed: Building local_dists.pex
12:30:02.89 [INFO] Completed: Building 48 requirements for lib.pex from the lockfiles/default.lock resolve: Cerberus==1.3.2, Fiona==1.8.20, Pillow==9.0.1, Shapely==1.8.5, aiobotocore[boto3]==2.5.0, aiohttp==3.8.3, asgiref==3.6.0, ... (1097 characters truncated)
Traceback (most recent call last):
  File "/tmp/named_caches/pex_root/venvs/a3d560e19ae43c8966072dc55452811abfe304b9/5fd7049af63e03f347278c89401424cd9731df9a/pex", line 274, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/usr/local/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/nefino_li/django/manage.py", line 12, in <module>
    run_manage()
  File "/nefino_li/django/manage.py", line 7, in run_manage
    from django.core.management import execute_from_command_line
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/__init__.py", line 1, in <module>
    from django.utils.version import get_version
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/utils/version.py", line 6, in <module>
    from distutils.version import LooseVersion
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 914, in _find_spec
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 97, in find_spec
    return method()
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 108, in spec_for_distutils
    mod = importlib.import_module('setuptools._distutils')
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 7, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 77, in do_override
    ensure_local_distutils()
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 63, in ensure_local_distutils
    core = importlib.import_module('distutils.core')
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 22, in <module>
    from .dist import Distribution
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 26, in <module>
    from .fancy_getopt import FancyGetopt, translate_longopt
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/setuptools/_distutils/fancy_getopt.py", line 14, in <module>
    import getopt
  File "/usr/local/lib/python3.8/getopt.py", line 38, in <module>
    from gettext import gettext as _
  File "/usr/local/lib/python3.8/gettext.py", line 74, in <module>
    _token_pattern = re.compile(r"""
  File "/usr/local/lib/python3.8/re.py", line 252, in compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python3.8/re.py", line 304, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/local/lib/python3.8/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/lib/python3.8/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/local/lib/python3.8/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.8/sre_parse.py", line 839, in _parse
    state.closegroup(group, p)
  File "/usr/local/lib/python3.8/sre_parse.py", line 97, in closegroup
    self.groupwidths[gid] = p.getwidth()
  File "/usr/local/lib/python3.8/sre_parse.py", line 184, in getwidth
    l, h = av.getwidth()
  File "/usr/local/lib/python3.8/sre_parse.py", line 198, in getwidth
    i, j = av[2].getwidth()
  File "/usr/local/lib/python3.8/sre_parse.py", line 179, in getwidth
    for op, av in self.data:
KeyboardInterrupt
Traceback (most recent call last):
  File "/tmp/named_caches/pex_root/venvs/a3d560e19ae43c8966072dc55452811abfe304b9/5fd7049af63e03f347278c89401424cd9731df9a/pex", line 274, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/usr/local/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/nefino_li/django/manage.py", line 12, in <module>
    run_manage()
  File "/nefino_li/django/manage.py", line 9, in run_manage
    execute_from_command_line(args)
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 363, in execute
    settings.INSTALLED_APPS
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 69, in _setup
    self._wrapped = Settings(settings_module)
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/nefino_li/django/django_core/settings.py", line 11, in <module>
    from nefino_common.aws import Boto3Config
  File "/nefino_li/libs/common/nefino_common/aws.py", line 7, in <module>
    import boto3
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/boto3/__init__.py", line 16, in <module>
    from boto3.compat import _warn_deprecated_python
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/boto3/compat.py", line 19, in <module>
    from boto3.exceptions import PythonDeprecationWarning
  File "/tmp/named_caches/pex_root/venvs/s/27ff5ef1/venv/lib/python3.8/site-packages/boto3/exceptions.py", line 15, in <module>
    import botocore.exceptions
KeyboardInterrupt
Here is the BUILD file for django:
Copy code
python_sources(
    name="lib",
    dependencies=[
        "//:root#psycopg2-binary",
        "//:root#openai",
        "django/cms/wagtail_hooks.py",
        "django/iris:iris",
        ":templates",
        ":translations",
        "django/cms/templatetags:cms-templatetags",
        "django/energy/templatetags:energy-templatetags",
        "django/django_core:django_core",
    ],
    overrides={"manage.py": {"restartable": env("USE_LOCALSTACK") == "true"}},
    run_goal_use_sandbox=False,
)
@happy-kitchen-89482 I had the suspicion that it is related to my changes here https://github.com/pantsbuild/pants/issues/17851. But not mounting the 
tmp
folder anymore did not resolve the issue. Ideas?
h
How does docker factor in here?
t
Pants is running within a VSCode devcontainer to which the hosts /tmp folder is mounted. Maybe there is constant changes in that temp folder which might affect pants?!
@happy-kitchen-89482 When I comment out the 
overrides
property, it does work. Ideas?
h
Hm, interesting.
What happens if you hardcode 
"restartable": True
there instead of commenting out?
b
I'll chip in. It's the same behaviour, but also in AWS. We have a sidecar container that runs migrations and it will show the same error behaviour with the 
KeyboardInterrupt
in the AWS CloudWatch logs.
t
@happy-kitchen-89482 This happens quite randomly on different working stations, but same code 😮
Open in Slack
PreviousNext
Powered by