std.provenance

Markdown Open in ChatGPT Open in Claude

std.provenance keeps citations honest when you assemble one answer from many sources. Each unique source URL gets a stable global id in first-seen order, and each document’s local [n] markers are rewritten to those global ids — so a merged report cites [1], [2], [3] consistently, no matter what each source called them.

from std.provenance import Cit, Doc, build_url_to_id, rewrite

Why it exists

When you combine several model outputs, their per-document [1], [2] markers collide and lose meaning. Reconciling them into one global bibliography is fiddly bookkeeping (typically a _build_url_to_id_mapping + _rewrite_text_with_global_ids pair). std.provenance standardizes it so citations survive assembly — the traceability half of provenance. For the broader concept, see Provenance.

The types

A Cit is a citation span pointing at a URL, with start/end character offsets into its document’s text. A Doc is text plus its citations:

d = Doc(
    text="Edge inference cuts latency [1] and improves privacy [2].",
    citations=[Cit(url="https://a.example", start=27, end=30),
               Cit(url="https://b.example", start=52, end=55)],
)

Assigning global ids

build_url_to_id(docs) scans all documents and returns a dict mapping each unique URL to a stable id, numbered in first-seen order across the whole set:

ids = build_url_to_id([doc_a, doc_b])
# {"https://a.example": 1, "https://b.example": 2, ...}

Rewriting markers

rewrite(text, cits, ids) replaces each citation span in a document’s text with its global [id], assembled left-to-right. It sorts spans by start offset first (via the module’s by_start), so it is robust to citation order:

ids = build_url_to_id([doc_a, doc_b])
merged_a = rewrite(doc_a.text, doc_a.citations, ids)
merged_b = rewrite(doc_b.text, doc_b.citations, ids)
# Both now reference the same global [1], [2], … numbering.

See the full type and function reference in the generated std.provenance API reference.