The specification curve — audit

Cells ordered by their estimate, with the grid coordinates alongside – the table behind the classic specification-curve figure. With ggplot2 installed and plot = TRUE, the estimates are drawn against their rank, with a reference line at zero. The companion specification panel (which cell sits at each rank) is not drawn; the returned tibble carries the rank together with every grid column, so you can lay out that panel yourself or read off which specification produced any point.

Usage

audit_curve(audit, plot = FALSE)

Arguments

audit: An audit_run() result.
plot: Draw the figure (needs ggplot2); default FALSE.

Value

The ordered tibble (the estimate ranking plus the grid coordinates), returned visibly whether or not the figure is drawn.

Examples

plan <- audit_plan(data.frame(text = c("a", "b")), "text",
  function(d) mean(d$label == "yes"), c("yes", "no"), "{text}")
plan <- audit_add_models(plan,
  list(demo = LLMR::llm_config("groq", "demo", temperature = 0)))
runner <- function(x, ...) transform(x, response_text = c("yes", "no"))
audit <- audit_run(plan, .runner = runner)
audit_curve(audit)