dedupe is worse than we thought