It's 2013 -- why do we still have problems with hotspots in Bigtable/Kansas?
I have not looked into the backend design, but this seems like something that's been solved in the past.
Do we not have
- Dynamic-sized replicas (add more read-only replicas for hot keys)?
- What about adaptive sharding/rebalancing (rehash based on usage)?
- Hierarchical storage? (automatically move dormant accounts to older/slower hardware, move back when reactivated)
- Or... the ability to manually move a single hot key to it's own instance. (variant of adaptive sharding above)
I know about the stubby cacher and how that can help, but that seems like treating the sympton rather than the source of pain.
#googplus