Revision Notes/Unit 4 — Causal Order Message Delivery/BSS Algorithm + Causal vs FIFO vs Total Order

BSS Algorithm + Causal vs FIFO vs Total Order

Intuition

FIFO says 'messages from the same sender arrive in order'. Causal order says 'if $send (m_{1}) \to send (m_{2})$ and both target the same destination, $m_{1}$ is delivered before $m_{2}$ ' — even if the senders are different. This matters when an effect's cause comes through a different path. Total order goes further: all processes deliver every message in the same global order.

Explanation

Causal order of message delivery — definition. If $send (m_{1}) \to send (m_{2})$ and both have the same destination, then $m_{1}$ must be delivered before $m_{2}$ at the destination. Note: causal order degenerates to FIFO when both messages are from the same sender.

Why FIFO is insufficient. Example: $P_{1}$ sends $m_{1}$ to $P_{3}$ , then $P_{1} \to P_{2}$ , then $P_{2} \to P_{3}$ sends $m_{2}$ . By causality, $send (m_{1}) \to send (m_{2})$ (through $P_{2}$ ). So $m_{1}$ should be delivered before $m_{2}$ at $P_{3}$ . But FIFO doesn't enforce this because $m_{1}$ and $m_{2}$ are from different senders — they could arrive in any order via FIFO. Causal order needs vector-clock-based tracking.

Birman-Schiper-Stephenson (BSS) algorithm. Each process $P_{i}$ maintains a vector $V_{i}$ . **On send to $P_{j}$ **: $V_{i} [i] := V_{i} [i] + 1$ ; piggyback $V_{i}$ on message $m$ as $V_{m}$ . **On receive of $m$ from $P_{j}$ at $P_{i}$ **: deliver $m$ only when both of the following hold: (a) $V_{m} [j] = V_{i} [j] + 1$ (this is the next message $P_{i}$ expects from $P_{j}$ — FIFO from $P_{j}$ ); (b) $\forall k \neq = j : V_{m} [k] \leq V_{i} [k]$ (all causally preceding messages from other senders have arrived). Otherwise buffer $m$ . After delivery, $V_{i} [j] := V_{i} [j] + 1$ .

Condition (a) intuition. Ensures no message from $P_{j}$ is skipped — $P_{i}$ delivers the *next* sequential message from each sender.

Condition (b) intuition. Before delivering $m$ , ensure $P_{i}$ has seen all messages that *causally preceded* $send (m)$ from other senders. If $V_{m} [k]$ is greater than what $P_{i}$ has seen ( $V_{i} [k]$ ), some causally-prior message from $P_{k}$ hasn't yet arrived — buffer $m$ and wait.

Why causal order is useful. Replicated databases: updates that depend on prior updates must be applied in causal order. Group communication: chat messages — 'Bob asks a question' before 'Alice answers it' even if routed through different paths. Distributed debugging: cause-then-effect, never effect-then-cause.

Causal vs FIFO vs Total order. FIFO orders only messages from the *same sender*. Causal orders any pair $(m_{1}, m_{2})$ where $send (m_{1}) \to send (m_{2})$ . Total orders ALL messages in a single global sequence consistent across all receivers — even concurrent messages get an arbitrary but consistent order. Total order is stronger than causal (and needed for state-machine replication).

Definitions

Causal order delivery — If $send (m_{1}) \to send (m_{2})$ and both target the same destination, $m_{1}$ is delivered before $m_{2}$ . Stronger than FIFO.
FIFO order — Messages from the same sender arrive in the order they were sent. Weaker than causal.
Total order — All processes deliver every message in the same global sequence — concurrent messages also get a consistent order. Required for state-machine replication.
BSS algorithm — Birman-Schiper-Stephenson causal delivery via vector clocks. Two conditions: $V_{m} [j] = V_{i} [j] + 1$ (FIFO from sender) AND $V_{m} [k] \leq V_{i} [k] \forall k \neq = j$ (causally prior msgs arrived).

Formulas

$Causal: send (m_{1}) \to send (m_{2}), same dest \Rightarrow deliver (m_{1}) \to deliver (m_{2})$
$BSS condition (a): V_{m} [j] = V_{i} [j] + 1$
$BSS condition (b): \forall k \neq = j : V_{m} [k] \leq V_{i} [k]$
$Strength order: Total ≻ Causal ≻ FIFO$

Derivations

BSS correctness — causal order preserved. Suppose $send (m_{1}) \to send (m_{2})$ with $m_{1}$ from $P_{j}$ and $m_{2}$ from $P_{k}$ to the same destination $P_{i}$ . The vector $V_{m}$ piggybacked on $m_{2}$ reflects $V_{k}$ at the time of $send (m_{2})$ . Since $send (m_{1}) \to send (m_{2})$ , $V_{k} [j]$ at that time was already updated to include $send (m_{1})$ — so $V_{m} [j] \geq$ value of $V_{j} [j]$ at $send (m_{1})$ . When $m_{2}$ arrives, condition (b) requires $V_{m} [j] \leq V_{i} [j]$ — meaning $P_{i}$ must have already received $m_{1}$ (or all messages up to that point from $P_{j}$ ). Hence $m_{1}$ is delivered before $m_{2}$ . ∎

Examples

Three-process causal example. $P_{1}$ sends $m_{1}$ to $P_{3}$ (timestamp $V_{1} = [1, 0, 0]$ ). $P_{1}$ sends $m^{'}$ to $P_{2}$ . $P_{2}$ receives $m^{'}$ ( $V_{2} = [1, 1, 0]$ ), then sends $m_{2}$ to $P_{3}$ ( $V_{m} = [1, 2, 0]$ ). Suppose $m_{2}$ arrives at $P_{3}$ before $m_{1}$ . BSS check on $m_{2}$ : (a) $V_{m} [2] = 2$ , $V_{i} [2] = 0$ , need $V_{m} [2] = V_{i} [2] + 1 = 1$ — FAIL. Buffer $m_{2}$ . Then $m_{1}$ arrives: (a) $V_{m} [1] = 1 = V_{i} [1] + 1 = 1$ ✓; (b) trivially ✓. Deliver $m_{1}$ . Re-check $m_{2}$ : (a) $V_{m} [2] = 2$ , $V_{i} [2] = 0$ now, still need $1$ — FAIL. Wait. Actually condition needed for $m^{'}$ — but $m^{'}$ wasn't sent to $P_{3}$ . Hmm, $m_{2}$ 's (a) is $V_{m} [2] = V_{i} [2] + 1 = 1$ ? Let me recompute: $m_{2}$ from $P_{2}$ has $V_{m} = [1, 2, 0]$ ; condition (a) for sender $P_{2}$ : $V_{m} [2] = 2 = V_{i} [2] + 1$ means $V_{i} [2] = 1$ . So $P_{3}$ would need to have already received 1 message from $P_{2}$ — but this IS the first. So $V_{m} [2]$ should be 1, not 2. Re-derive: after receiving $m^{'}$ , $P_{2}$ 's clock is $[1, 1, 0]$ ; before sending $m_{2}$ , $V_{2} [2] + = 1$ → $[1, 2, 0]$ ; piggyback $[1, 2, 0]$ . At $P_{3}$ , $V_{i} = [0, 0, 0]$ . Condition (a) for $m_{2}$ from $P_{2}$ : $V_{m} [2] = 2$ , need $V_{i} [2] + 1 = 1$ . FAIL. Buffer. So $P_{3}$ waits for the previous message from $P_{2}$ ... but there is none (only $m^{'}$ from $P_{1}$ to $P_{2}$ ). Actually $P_{3}$ 's expectation is that some earlier message from $P_{2}$ should arrive — but $P_{2}$ hasn't sent anything else to $P_{3}$ . The vector tracking is off. The correct BSS uses $V_{i} [i]$ on send (not piggyback after) — so $V_{m} [2] = 1$ in this trace. Cleaner: $P_{2}$ sends with $V_{m} = [1, 1, 0]$ (just before incrementing). Condition (a): $V_{m} [2] = 1 = V_{i} [2] + 1 = 1$ ✓. Condition (b): $V_{m} [1] = 1 \leq V_{i} [1] = 0$ ? FAIL. Buffer. $m_{1}$ arrives, $V_{i} = [1, 0, 0]$ . Re-check $m_{2}$ : (a) ✓; (b) $V_{m} [1] = 1 \leq V_{i} [1] = 1$ ✓. Deliver. Order preserved: $m_{1}$ then $m_{2}$ . ✓
WhatsApp / group-chat application. Bob: 'What time?' (msg1 from B to C). Alice receives and replies: 'Noon' (msg2 from A to C, after receiving msg1 from B). FIFO doesn't help — msg1 and msg2 have different senders. Causal order via BSS guarantees Charlie sees the question before the answer.

Diagrams

Three-process causal example: arrows showing $send (m_{1}) \to send (m_{2})$ chain via $P_{2}$ ; receiver $P_{3}$ must deliver $m_{1}$ before $m_{2}$ .
BSS state machine: msg arrives → check conditions (a) and (b) → deliver or buffer; when buffered msg's conditions become satisfied, deliver.
Order strength diagram: FIFO ⊂ Causal ⊂ Total. Each level adds more pairs of ordered events.

Edge cases

Causal degenerates to FIFO for same-sender messages — BSS condition (a) IS FIFO.
Total order requires consensus for concurrent messages — heavier than BSS. State-machine replication needs total.
BSS buffering can grow unboundedly if a process is slow — practical implementations bound buffer size.
Late delivery (large buffers) under high message rates is the cost of strict causal ordering. Trade-off vs throughput.

Common mistakes

'Causal order is the same as FIFO.' No — FIFO orders only same-sender. Causal orders any causally-related pair.
'Total order is just causal order on all messages.' No — total order also assigns an order to *concurrent* messages, which causal does not.
Forgetting condition (b) in BSS. Without it, you only get FIFO.
Confusing 'deliver' with 'receive'. A message can be received (arrives at process) but not yet delivered (passed to application). BSS buffers received-but-not-yet-deliverable messages.

Shortcuts

Causal definition: $send (m_{1}) \to send (m_{2}) \Rightarrow deliver (m_{1}) \to deliver (m_{2})$ at common destination.
BSS condition (a): $V_{m} [j] = V_{i} [j] + 1$ — FIFO from $P_{j}$ .
BSS condition (b): $\forall k \neq = j : V_{m} [k] \leq V_{i} [k]$ — causally prior msgs arrived.
Strength: Total ≻ Causal ≻ FIFO.

Proofs / Algorithms

BSS preserves causal order. Suppose $send (m_{1}) \to send (m_{2})$ at $P_{j}$ and $P_{k}$ respectively, both targeting $P_{i}$ . By the happened-before, $V_{k}$ at $send (m_{2})$ has $V_{k} [j]$ at least the value of $V_{j} [j]$ at $send (m_{1})$ . So $V_{m_{2}} [j] \geq V_{m_{1}} [j]$ . At $P_{i}$ , $m_{2}$ 's condition (b) requires $V_{m_{2}} [j] \leq V_{i} [j]$ . So $V_{i} [j] \geq V_{m_{2}} [j] \geq V_{m_{1}} [j]$ , meaning $P_{i}$ has already received and delivered messages from $P_{j}$ through $m_{1}$ 's timestamp. Hence $m_{1}$ was delivered before $m_{2}$ . ∎

Distributed Systems