Add measurement perturbations — audit_add_perturbations • LLMRcontent

Add measurement perturbations

Usage

audit_add_perturbations(plan, label_order = NULL, temperature = NULL)

Arguments

plan: An audit_plan().
label_order: "as_given", "reversed", or both. With LLMs the order in which options are listed is a real effect, not a nuisance; auditing it is the point.
temperature: Numeric vector of temperatures to cross (e.g. c(0, 0.7)).

Value

The plan, extended.

Examples

plan <- audit_plan(data.frame(text = c("a", "b")), "text",
  function(d) mean(d$label == "yes"), c("yes", "no"), "{text}")
audit_add_perturbations(
  plan, label_order = "reversed", temperature = 0.7)