Cardano's incident between block 8300569 and 8300570
About the incident between block 8300569 and 8300570) which caused approx. 50% of Cardano nodes to disconnect and restart.. ... Analysis of 'someone ' I’ve been debugging this for a few hours now and have a fairly strong opinion that the issue is indeed a bug in the balanced tree algorithm implementation of the Haskell containers package. Specifically a logic error that presumes one of the container subtree bins can never have a left or right sub-sub-tree node that is empty when attempting to rebalance the parent. Probably a condition that arises when a Map is being aggressive purged of values in successive calls, resulting in an unbalanced branch and violation of the assumed invariant here: https://github.com/haskell/containers/blob/v0.6.5.1/containers/src/Data/Map/Internal.hs#L4154 Rather than attempting to recreate the conditions by capturing another sequence of TXs, it might be easier to just code a stress harness around the Map container of the Haskell/containers package