sharding

term

cloudprogramming

Splitting a dataset across machines by some key, so each holds part of it rather than all of it.

It is what you do when one machine can no longer hold or serve the data, and the shard key is the decision that cannot be undone cheaply. Choose badly and traffic concentrates on one shard while the others idle, or the common query has to visit every shard and the whole point is lost. Everything hard about it - rebalancing, cross-shard transactions, hot keys - descends from that single early choice.

Also known as: partitioning

All glossary entries