Package index • LLMRcontent

Codebooks and gold standards

codebook(): Create a codebook
gold_set(): Build a gold-standard set with split provenance and a sealed holdout split
protocol(): Assemble a coding protocol
cb_category(): Define one category of a codebook
codebook_labels(): Labels defined by a codebook
codebook_hash(): Content hash of a codebook
format_codebook(): Render a codebook as prompt text
gold_split(): Rows of a gold set belonging to one split
gold_ledger(): The evaluation ledger of a sealed gold set
gold_size(): Plan the size of a gold set
coder_agreement(): Agreement among coders or replicates

tune_protocol(): Tune candidate protocols on the development split
protocol_lock(): Lock a protocol
validate_protocol(): Validate a locked protocol on the sealed holdout split
code_corpus(): Code a corpus with a locked protocol
gold_correct(): Correct corpus prevalences with the gold-set audit

audit_plan(): Declare a measurement-robustness audit
audit_add_models(): Add the model arm of the grid
audit_add_prompts(): Add prompt paraphrases
audit_add_perturbations(): Add measurement perturbations
audit_run(): Run the audit grid
audit_units(): The unit-level trail behind an audit
audit_stability(): Stability of the estimate across the grid
audit_fragility(): How small a measurement change flips the conclusion?
audit_curve(): The specification curve
audit_placebo(): Placebo tests for the measurement pipeline
sign_flip() threshold_flip(): Conclusion-flip rules for the fragility index

archive_build(): Build a replication archive from an LLMR audit log
archive_seal(): Seal an archive under a root hash
archive_check(): Verify an archive's integrity and (optionally) a result's completeness
archive_redact(): Redact an archive's content while keeping its hash tree
archive_replay(): Replay archived responses offline
archive_drift(): Measure drift in archived calls against the live providers
archive_write(): Write an archive to a directory
archive_read(): Read an archive from a directory
reset(<archive_replayer>): Reset an archive replayer to its initial position

as_tibble(<coded_corpus>): Coerce a coded corpus to a tibble
as_tibble(<protocol_tuning>): Coerce protocol tuning results to a tibble
diagnostics(<protocol_validation>): Protocol-validation diagnostics
report(<protocol_validation>): Report a protocol validation through the LLMR generic
as_tibble(<gold_correction>): Coerce a gold correction to a tibble
diagnostics(<gold_correction>): Gold-correction diagnostics
as_tibble(<archive>): Convert an archive manifest to a tibble
diagnostics(<archive>): Machine-readable archive diagnostics
report(<archive>): Draft an archive report through the shared LLMR generic
as_tibble(<audit>): Coerce an audit to a tibble
as_tibble(<audit_placebo>): Coerce an audit placebo to a tibble
diagnostics(<audit>): Audit diagnostics through LLMR's shared generic
report(<audit>): Report an audit through LLMR's shared generic