Hash-Chaining KDE Records for Tamper-Evidence in Python

The concrete task here is narrow and common: you have a stream of FSMA 204 Key Data Element (KDE) records landing in an append-only table, and you need to make them tamper-evident so that any after-the-fact edit is provable, not invisible. The failure this prevents is the silent mutation — someone runs an UPDATE on a stored Cooling event, or a botched migration rewrites a traceability_lot_code, and nothing in the data reveals that the record no longer matches what was originally submitted. This guide implements per-record hash chaining in Python — sha256(prev_hash + canonical_payload) for every row — and then walks the finished chain to detect exactly which record was tampered with. It builds directly on the design in the Hash-Chained Ledger guide; read that first for the ledger schema and the append transaction.

The reason to care is regulatory. 21 CFR 1.1455(a) requires that retained records be accurate and unaltered, and the FDA can ask you to reproduce a lot’s KDEs within twenty-four hours. If you cannot demonstrate that the records you hand over are byte-for-byte what you recorded, you have satisfied retention but not integrity — and an auditor has no basis to trust the trace.

Root Cause: Independent Rows Have No Memory of Each Other

A plain audit table stores each KDE record in isolation. Row 42 knows nothing about row 41, so editing row 42 leaves row 41 — and every other row — completely undisturbed. There is no structural relationship between records, which means there is nothing to break when one is altered. That is the root cause of undetectable tampering: independence.

Hash chaining removes the independence on purpose. Each record carries the hash of the record before it, so the rows stop being a set and become a sequence in which every element cryptographically depends on its predecessor. Editing one payload changes that payload’s hash; the next row’s stored prev_hash no longer matches the recomputed value; and because that next row’s own hash was computed from its prev_hash, its hash is now wrong too, and the invalidation cascades forward to the head. The tampering that was invisible in an independent table is now a break you can walk to and point at. The one subtlety that trips people up is canonicalization: if the bytes you hash on write differ from the bytes you hash on verify — because JSON keys reordered or a Decimal serialized differently — the chain will report a break that is really just a serialization mismatch. Getting the canonical form exactly right is most of the work.

Minimal Reproducible Example

The example below builds a three-record chain from realistic Shipping and Receiving KDEs, then mutates one stored payload to simulate tampering. Each record’s hash is sha256(prev_hash + canonical_json(payload)). Run it and the naive “does the head look right” check passes even after tampering — because the attacker who edits a payload can trivially also not touch the head — which is exactly why a single stored head is not enough on its own.

import hashlib
import json
from decimal import Decimal

GENESIS = "0" * 64


def canonical(payload: dict) -> bytes:
    """Deterministic bytes: sorted keys, no whitespace, Decimals as strings."""
    def default(o):
        if isinstance(o, Decimal):
            return str(o)
        raise TypeError(f"not serializable: {type(o)}")
    return json.dumps(payload, sort_keys=True, separators=(",", ":"),
                      default=default).encode("utf-8")


def record_hash(prev_hash: str, payload: dict) -> str:
    body = prev_hash.encode("utf-8") + canonical(payload)
    return hashlib.sha256(body).hexdigest()


# Three KDE records as they were originally submitted.
records = [
    {"record_id": "r-5001", "cte": "Shipping", "lot": "LOT-A19",
     "gln": "0614141000012", "qty": Decimal("40.0"), "uom": "CASE"},
    {"record_id": "r-5002", "cte": "Receiving", "lot": "LOT-A19",
     "gln": "0614141999996", "qty": Decimal("40.0"), "uom": "CASE"},
    {"record_id": "r-5003", "cte": "Shipping", "lot": "LOT-B27",
     "gln": "0614141000012", "qty": Decimal("18.5"), "uom": "CASE"},
]

# Build the chain: each row stores prev_hash and its own record_hash.
chain = []
prev = GENESIS
for rec in records:
    h = record_hash(prev, rec)
    chain.append({**rec, "prev_hash": prev, "record_hash": h})
    prev = h

head = chain[-1]["record_hash"]

# Tamper: quietly change the received quantity on row r-5002.
chain[1]["qty"] = Decimal("4.0")   # 40 cases become 4 — a shrinkage cover-up

# Naive check: "is the stored head still the value we remember?" — useless,
# because tampering a payload does not touch the separately-stored head field.
print("stored head unchanged:", chain[-1]["record_hash"] == head)  # True — misleading

The final print is True, which is the trap: the head field was not edited, so comparing it to the remembered head detects nothing. The payload was corrupted anyway. Tamper-evidence has to come from re-deriving the chain, not from trusting a stored hash.

Fix: Re-Walk the Chain and Localize the Break

The fix is a verifier that ignores the stored record_hash values as inputs and recomputes them from the payloads, checking two invariants at each step: the stored prev_hash must equal the previous row’s recomputed hash, and the row’s stored record_hash must equal its recomputed hash. The first row whose recomputation disagrees is the first tampered record. Comparing the recomputed head against an independently-sealed trusted head (from a daily root, per the parent guide) is what makes the whole result trustworthy.

from dataclasses import dataclass


@dataclass
class ChainBreak:
    index: int
    record_id: str
    reason: str


def verify_chain(chain: list[dict], trusted_head: str | None = None) -> ChainBreak | None:
    """Re-derive every hash from payloads; return the first divergence or None.

    trusted_head is an externally-sealed value (e.g. a signed daily root). If the
    chain internally verifies but the recomputed head differs from trusted_head,
    the tail was rewritten wholesale and re-linked — still caught.
    """
    prev = GENESIS
    for i, row in enumerate(chain):
        payload = {k: row[k] for k in row
                   if k not in ("prev_hash", "record_hash")}
        if row["prev_hash"] != prev:
            return ChainBreak(i, row["record_id"], "prev_hash mismatch")
        recomputed = record_hash(prev, payload)
        if recomputed != row["record_hash"]:
            return ChainBreak(i, row["record_id"], "record_hash mismatch")
        prev = recomputed

    if trusted_head is not None and prev != trusted_head:
        return ChainBreak(len(chain) - 1, chain[-1]["record_id"],
                          "head does not match sealed daily root")
    return None


break_found = verify_chain(chain, trusted_head=head)
if break_found:
    print(f"TAMPER at index {break_found.index} "
          f"({break_found.record_id}): {break_found.reason}")
else:
    print("chain verified: all records unaltered")

The key move is that verify_chain never trusts a payload’s own record_hash as authority — it recomputes it. On the tampered chain, row r-5002’s payload now hashes differently, so its recomputed record_hash diverges from the stored one, and the verifier stops and names it. The chain has memory again.

Verification Steps

Confirm the fix with three checks that mirror what an auditor would do:

1. Build the clean chain and assert verify_chain(chain) is None — an untouched chain verifies.
2. Mutate any single payload field and assert the returned ChainBreak.record_id equals the row you edited — tampering is localized, not just detected.
3. Rewrite the entire tail (edit a payload and re-derive downstream hashes so the chain is internally consistent again) and assert that verify_chain(chain, trusted_head=sealed_head) still returns a break at the head — the sealed daily root catches a wholesale rewrite that an internal-only check would miss.

# 1. clean chain verifies
clean = build_clean_chain(records)
assert verify_chain(clean) is None

# 2. single-field tamper is localized
clean[1]["qty"] = Decimal("4.0")
b = verify_chain(clean)
assert b is not None and b.record_id == "r-5002"

# 3. re-linked tail is still caught by the sealed head
relinked = rebuild_from(clean, start=1)   # re-hash rows 1..n after the edit
assert verify_chain(relinked, trusted_head=sealed_head) is not None
print("all tamper-evidence assertions passed")

In production the same three assertions run as a scheduled job over the live ledger, logging a structured line per verification pass and paging an operator the moment a ChainBreak is returned. A passing run is the evidence you show an auditor that the KDE records they are looking at are the ones you wrote.

Related Edge Cases

Three variants are worth checking next. First, timestamp normalization: if event_timestamp is stored in local time on write but compared in UTC on verify, canonicalization produces different bytes and the chain falsely breaks — always normalize to UTC before hashing. Second, Decimal precision drift: Decimal("40.0") and Decimal("40") serialize differently, so a quantity that round-trips through a float somewhere will silently change its hash; keep quantities as strings end to end. Third, concurrent appends forking the chain: two writers that both read the same head will produce two rows with the same prev_hash, which the verifier sees as a gap — serialize appends and check for duplicate prev_hash values, a topic the Data Retention Policies sweep also guards against.

Frequently Asked Questions

Related

• Hash-Chained Ledger — the append-only ledger design and schema this walk-through implements
• Verifying Ledger Integrity with Merkle Proofs — prove one record’s membership without re-walking the whole chain
• Data Retention Policies — the scheduled sweep that runs this verification before any purge
• Security Boundaries for Trace Data — ensures only authenticated, validated records ever enter the chain
• Audit-Log Export — packages verified records and their hashes for an FDA request

Up: Hash-Chained Ledger — this guide is the hands-on Python implementation of that ledger’s tamper-evidence property.