Pants main 1st reaction a bit of horror Has anyone seen this

Question

Pants main 1st reaction a bit of horror Has anyone seen this ```$ pants check src python pants engine 11 45 13 52 ERROR Completed Typecheck using MyPy mypy mypy failed exit code 1 src python pants engine collection py note In class Collection src python pants engine collection py 55 16 error Incompatible types in assignment expression has type Callable int base class object defined the type as Callable object int assignment hash = Tuple hash ^ src python pants engine target py note In class CoarsenedTargets src python pants engine target py 917 16 error Incompatible types in assignment expression has type Callable int base class object defined the type as Callable object int assignment hash = Tuple hash ^ Found 2 errors in 2 files checked 69 source files mypy failed ```

enough-analyst-54434 · Answer

Edit the 1st file like so ```$ git diff diff git a src python pants engine collection py b src python pants engine collection py index c340fa2e6 9f3bb81c2 100644 a src python pants engine collection py +++ b src python pants engine collection py 52 7 +52 8 class Collection Tuple T Unlike in Python 2 we must explicitly implement hash since we explicitly implement eq per the Python 3 data model See <https docs python org 3 reference datamodel html object hash > hash = Tuple hash + def hash self gt int + return super hash def repr self gt str return f self class name list self ```

enough-analyst-54434 · Answer

And then horror ``` $ pants check src python pants engine 11 46 15 08 ERROR 1 Exception encountered Engine traceback in select in pants core util rules environments determine local environment in pants core util rules environments determine all environments Traceback most recent call last File home jsirois dev pantsbuild jsirois pants src python pants util frozendict py line 91 in calculate hash return hash tuple self data items File home jsirois dev pantsbuild jsirois pants src python pants engine collection py line 56 in hash super hash File lt string gt line 3 in hash TypeError unhashable type dict During handling of the above exception another exception occurred Traceback most recent call last File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants core util rules environments py line 592 in determine all environments Get EnvironmentTarget EnvironmentName name for name in environments subsystem names File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 358 in MultiGet return await MultiGet tuple arg0 File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 165 in await result = yield self gets File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants core util rules environments py line 827 in get target for environment name WrappedTargetRequest address description of origin= description of origin File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 118 in await result = yield self File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants engine internals graph py line 401 in resolve target for bootstrapping description of origin=request description of origin File home jsirois dev pantsbuild jsirois pants src python pants engine internals graph py line 180 in determine target adaptor and type TargetAdaptorRequest address description of origin=description of origin File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 118 in await result = yield self File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants engine internals build files py line 395 in find target adaptor address family = await Get AddressFamily AddressFamilyDir address spec path File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 118 in await result = yield self File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants engine internals build files py line 395 in find target adaptor address family = await Get AddressFamily AddressFamilyDir address spec path File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 118 in await result = yield self File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants engine internals build files py line 260 in parse address family Get SyntheticAddressMaps SyntheticAddressMapsRequest directory path File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 509 in MultiGet return await MultiGet arg0 arg1 File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 165 in await result = yield self gets File home jsirois dev pantsbuild jsirois pants src python pants engine internals selectors py line 623 in native engine generator send res = rule send arg if err is None else rule throw throw or err File home jsirois dev pantsbuild jsirois pants src python pants engine internals synthetic targets py line 314 in all synthetic targets spec paths=set itertools chain from iterable all spec paths File home jsirois dev pantsbuild jsirois pants src python pants engine internals synthetic targets py line 255 in create sorted address maps key=address map key key=address map key File home jsirois dev pantsbuild jsirois pants src python pants util frozendict py line 49 in init self hash = self calculate hash File home jsirois dev pantsbuild jsirois pants src python pants util frozendict py line 104 in calculate hash TypeError Even though you are using a `FrozenDict` the underlying values are not hashable Please use hashable and preferably immutable types for the underlying values e g use tuples instead of lists and use FrozenOrderedSet instead of set ```

enough-analyst-54434 · Answer

Soo it seems 1 Somehow CI has been skipping this for a good long while 2 The ` hash ` override we though we had with a big comment as to why it was important we apparently didn t have

worried-painter-31382 · Answer

Just fyi for the ` hash ` the diff is missing a return statement grimacing

enough-analyst-54434 · Answer

Yeah added it but of course same issue The return was would ve never got hit in the backtrace

enough-analyst-54434 · Answer

Yeah this is a bit of a rabbit hole If you naively freeze up the backtrace objects you eventually hit `sources= ` BUILD file arguments and that s a whole can of worms mapping lists to tuples and dicts to FrozenDicts recursively It may just be that a less naive hash would work over some proxy I ll file an issue for someone more tuned in here to take a look at

enough-analyst-54434 · Answer

Ok the MyPy check failing locally but not in CI appears to be a Heisenbug I no longer see MyPy failures today So no clue but bug for another day I guess The bug it revealed though is real We currently hash every pants engine collection Collection to the same value and ditto pants engine target CoarsenedTarget Fixing that leads to more bad as described above a bunch of unhashable collected things at least in CoarsenedTarget <https github com pantsbuild pants issues 18250> I ll turn this over to others

happy-kitchen-89482 · Answer

Yikes