Gossip protocol is a style of inter-process communication in the network, inspired by gossips that happen in a human social groups. Basically, nodes (peers) exchange some information. This information may contain the knowladge that node gain by itself (eg some events), received from user (eg search queries) or received from other nodes during conversation with them.
Peers are connecting to each other in a random manner to ensure that information is shared pretty quickly. This could be like 10 times per second that one peer connects to another and exchanges the information.
It's also possible to organize the groups of hosts where each node share the information only with nodes within this group. This simplifies finding next random node to communicate, and limits the number of random useless connections between nodes. Some nodes are shared between multiple groups, to ensure that information is spread over the whole network. As an alternative, nodes could communicate only with it's near neighbors. Another alternative is to define the hierarchical levels, thus minimize the need to send message to everyone, but instead only some hosts in the specified level.
Gossip protocol unfortunately does not not guarantee that knowledge would be shared between all nodes. Another issue is that the received information might be not correct or valid anymore (well, that's in the nature of gossips, right?). Also it is said that gossip protocol maintains relaxed consistency.
Another good usage of gossips is to multicast information over the network without known structure. For example, it's not always possible to multicast information using multicast IP, because hosts might be organized into different restricted networks.
Gossip protocols are in the core of the internet.
When using gossip protocols it's better to share information about the fact than the fact itself, ie configuration was changed instead of new value for property A is B. This is also true in many distributed systems.
Usages
There are multiple usages for gossip protocol, some of them are:
- propagate events between multiple hosts, so each node could react if need
- could be used to route data using different paths (get the data, split on packages, deliver to the target using different routes)
- could be used to aggregate values among different hosts (each node receive the data, check if it was seen, if not - aggregate with own data and send forward to next hosts)
- search among large amount of nodes (used in unstructured P2P networks)
- finding the list of nodes or just counting nodes in the network of unknown size and structure
- electing leaders
- sorting nodes in the network, detecting failures and errors, recovering the network structure
- used for monitoring and gathering data, for example for load balancing
- can help with detecting network partitions and dynamic insertions of new services
- heartbeat implementation
- implementing epidemic algorithms in the network
- gather information about other nodes to make local decision, like load balancing, accessing data located in specific nodes etc.; do this in scalable way
- detecting new nodes and additing them to the cluster or load balancer, updating DNS records etc.
But usually gossip protocols are not the most efficient one. It's pretty often possible to find some specific protocol that does the job better. However, some other protocols may not work for large networks (ie paxos), while others are centralized thus might have failure issues (ie based on centralized db).
Another downside of gossip protocols are:
same information can be delivered same host twice
message latency is usually high especially for large networks
No comments:
Post a Comment