The verifiability horizon of an archive — verifiability

Classifies every model in the archive by how long its calls remain re-runnable:

Usage

verifiability_horizon(
  archive,
  open_patterns = "gpt-oss|llama|qwen|deepseek|mistral|mixtral|gemma|phi-|kimi|glm-|yi-"
)

Arguments

archive: An archive.
open_patterns: Regular expression matched (case-insensitively) against model names to classify them as open-weight.

Value

A tibble: model, provider, calls, class.

Details

"open-pinnable": open-weight families (re-runnable indefinitely against a pinned checkpoint; record the checkpoint hash in the paper).
"api-contingent": closed models behind a live API (re-runnable only while the provider serves this version; the archived model_version tells you which one to ask for).

Classification is a heuristic over provider and model names; override with open_patterns when you serve something unusual.

Examples

log <- tempfile(fileext = ".jsonl")
writeLines(c(
  paste0('{"ts":"2026-06-01T10:00:01+0000","schema_version":"1.0",',
    '"kind":"call","provider":"groq","model":"openai/gpt-oss-20b",',
    '"usage":{"sent":5,"rec":2},"response_id":"r-1"}'),
  paste0('{"ts":"2026-06-01T10:00:02+0000","schema_version":"1.0",',
    '"kind":"call","provider":"openai","model":"gpt-4o",',
    '"usage":{"sent":5,"rec":2},"response_id":"r-2"}')), log)
verifiability_horizon(archive_build(log))