Mypy Fun of the Day! (or Pants Caching Fun) I edi...
# development
f
Mypy Fun of the Day! (or Pants Caching Fun) I edited files in `src/python/pants/backend/python/providers/python_build_standalone`and am now getting seemingly unrelated type errors in existing Pants code:
Copy code
Partition #2 - python-default, ['CPython==3.11.*']:
src/python/pants/engine/collection.py: note: In class "Collection":
src/python/pants/engine/collection.py:55:16: error: Incompatible types in assignment (expression has type "Callable[[], int]", base class "tuple" defined the type as "Callable[[tuple[T, ...]], int]")  [assignment]
        __hash__ = Tuple.__hash__
                   ^~~~~~~~~~~~~~
src/python/pants/engine/target.py: note: In class "CoarsenedTargets":
src/python/pants/engine/target.py:939:16: error: Incompatible types in assignment (expression has type "Callable[[], int]", base class "tuple" defined the type as "Callable[[tuple[CoarsenedTarget, ...]], int]")  [assignment]
        __hash__ = Tuple.__hash__
                   ^~~~~~~~~~~~~~
h
Are you running
check
with
--changed-since=main
or something like that?
f
nope, just
./pants fmt lint check src/python/pants::
h
Is that on macos? I have seen issues like this sometimes, where mypy complains when I run it locally but passes in CI
f
Yes on macOS
just tried on linux, and same command passes, so wtf macOS?
h
Indeed!
c
f
seems spot on
b
I imagine it's weird to you also that that latent issue has suddenly started to be flagged, even without relevant code changes
f
seems like a "heisen-bug" at first glance
h
In a PR that didn’t touch anything related
But it seems like we should fix the underlying problem? I’m not sure why we haven’t
Anyway, it’s not MacOS…
f
The mutability is in TargetAdaptor, AddressMap, and SyntheticAddressMap.
Current version seems to fix it. Would love if someone could rebase on top of the branch and see if your PR passes
check
.
(or at least does not fail because of the
Collection
__hash__
stuff)
h
This has led me down a rabbit hole of cascading mutability bugs…
I will try and post some findings, once I figure out what the hell is going on
f
We've probably been fortunate that Pants has been working still even with these mutability bugs.
h
As far as I can tell we have been fortunate for two reasons: 1) We (mostly?) don’t use Collection subclasses as keys in dicts, so we don’t get pathological hash buckets. We still need them to be hashable because FrozenDict requires its values to be hashable, for reasons I am not totally clear on. 2) We’re relying on mutability to support the
_extend_synthetic
functionality, but it looks like we don’t actually use that anywhere. From inspection, that functionality appears to be completely broken, but the tests aren’t substantial enough to expose this, and since we don’t actually use it in non-test code, and it’s not documented anywhere, we’re getting away with it. I need to dig deeper into this, and write a test to expose the problem I believe I am seeing. But this is not just a performance issue but a correctness one.