Peer-to-Peer Gossip Dissemination
Fabric
Vita Bortnikov
Haifa Research Lab
New Hyperledger Architecture
 Separates Consensus and Peers that keep the ledger and execute chaincode
– Better support for confidentiality
– Increase in scalability of number participants and transaction throughput
– Needs to provide the same guarantees as consensus , but in the separated
architecture
 Peers are endorsing validating and committing transactions
– Peers keep the raw (validated) ledger and global state
 Consenters run consensus to provide atomic broadcast – order the transactions
2
Communication Pattern
Two main groups: Consenters and Peers
 Communication between Consenters: managed as parts of consensus
Peers Communication and communication from Consenters to Peers.
 Communication between submitter and endorsers
– Submitter sends transaction information to endorsers set and then waits for enough
endorsements in order to proceed
 Chaincode deployment
– Submitter – Send chaincode to all endorsers;
 Consenters deliver new transaction and checkpoints to all peers
– The new transaction needs to get to all peers with enough information to be validated
and verified, need also to support flexible validation policy
– Consenters checkpoints need to get from consenters to all peers
 State transfer
– Each peer needs a way to complete missing information, this could be either some
3
blocks or big chunks of data
Data Dissemination Fabric
 Goal: Create a Peer-to-Peer Dissemination Fabric that will guarantee keeping an
identical ledger in all peers. This consists of:
– Dissemination of new transaction blocks and checkpoints from Consenters
(committing peers)
– Continuous state synch between peers to make sure all peers get all the
ledger
 Peer-to-peer gossip fabric challenges
– A peer needs a way to verify that the transaction originated from the
consensus (need to clearly define its interaction with consensus)
– Need to support byzantine behavior of consenters and peers
– Peers dynamicity and elasticity: support easily addition and removeal of peers
– Scalability in number of peers and transaction throughput
 Basis for other communication in the system
– Submitters to endorsers
4
Peer Communication Fabric
Gossip
fabric
Peer
Peer
Peer
Committer
Committer
Peer
Deliver (block)
Peer
Consenter
Consenter
Consenter
Consenter
Committer
Committer
Peer
Deliver (block)
Committer
Committer
Peer
5
Peer
Peer
Transaction Dissemination Flow
 Consenters run consensus algorithm and create atomic broadcast of transactions
– deliver (Bl) – deliver block from each consenter, signed by this consenter
 Consenting peers
– Wait for f+1 deliver(Bl) commands to make sure it is right commands (f
number of Byzantine consenters)
– Validate transactions from the block and update the Validated Ledger
– Disseminate block to the rest of the Peers including signatures from f+1
Consenters
– At least p+1 Consenting peers need to initiate transaction dissemination to
guarantee liveness (p – number of Byzantine Peers)
 Dissemination Fabric
– Connects all peers
– Makes sure transaction got to all peers in the system and can be validated by
the peer
6
Gossip Fabric – High Level Design
 All peer are organized in the gossip
 Peers maintain membership information about all available peers
 Both ledger state and membership state are disseminated using the gossip fabric
 Gossip Protocol in a nutshell
– In each round the peer chooses K random peers from the peer set and
 Propagates its changes to those peers (push)
 Gets new information from those peers (pull)
 Anti-entropy gossip protocol
 No need to maintain fixed connectivity; very robust; Easy to implement; Tolerates crash
and byzantine failures
 Each update reaches all peers in O(log(N)) rounds
 Same protocol for new messages and “filling the hole”
– Different handling will be added for large chunk state transfer
7
Gossip Fabric – API and Interaction
Ledger
1 2 3 4 ….. n-1 n
getBlocks(start, end) getNextAvailable
Blocks()
lastBlock = n
Gossip Fabric
8
n+2
n+3 n+5
• Validated Ledger keeps
continuous ledger of
transactions
• Dissemination Layer –
transfers it the next
block
• Gossip part keeps the
blocks after (with gaps)
• Gossip part uses
getBlocks() to get blocks
for state transfer
Peer membership information
 Gossip will maintain full membership information about existing peers
– The information is needed in order to choose randomly K peers in the gossip round
 Membership information will be a PeerSet, that will be exchanged using the same anti-entropy gossip
algorithm
 Once a new peer joins, it needs to know some initial bootstrapSet to connect to peers that already
connected to other peers
 Each node keeps PeersSet of {peer, counter}, Counter is used by each peer to re-inform others about its
liveness. The peers that didn’t update the counter for a long period are considered to be dead.
 Some details of the algorithm
– Only information about new peer and changes in timestamp are propagated using gossip
– A peer periodically disseminates via gossip a self-signed Alive message
– Each peer decides by itself about disconnected/dead peers based on the timestamp: this is the
simplest way to provide byzantine tolerance
 Current approach: simple and accurate for joining peers, slow for failed peers
 Peer signs the membership message, so it can verified as sources form the peer
9
Plans and Next steps
 Initial Implementation of ledger dissemination (including state transfer) – Nov
2016
 Design support for partitioned ledger
 Performance and scalability analysis of initial implementation
 Next steps:
– Improve transaction dissemination and validation efficiency;
 Reduce message complexity and signatures
 Optimized performance for non-byzantine and low-byzantine environments
 Evaluate various overlay-based approaches
– Provide efficient communication for submitter-endorser communication and
validation policies
10
Open questions
 The current design assumes the raw ledger is disseminated
– To disseminated validated ledger we need either multi-sig after validation or
many messages – efficiency and latency challenges
– Need to add support for more flexible validation policy
 Signatures:
– Alternatives to multi-sig
 Size of the transactions
 Consenters are ledger sources
– Need to decide about policy when the ledger at consenters can be cut
11