Compares closed-item response shares with human benchmark shares supplied by the user. The result contains deviations for covered item-response pairs, the number of closed items covered, and nonresponse rates by item. The function does not alter responses or adjust response shares. Without a benchmark, response shares describe the configured model under the supplied personas, not a human population.
Arguments
- responses
A
panel_administer()result.- benchmark
A data frame with columns
item_id,response, andshare(human marginal proportions). Shares within an item should sum to 1; a deviation beyond rounding draws a warning.- benchmark_name
How the source should be cited in reports (e.g.
"ANES 2024 pilot").
Value
responses with its benchmark field set:
$table (per covered item and response: share_silicon,
share_human, deviation), $nonresponse (nonresponse and execution
failure rates per item),
$items_covered / $items_total, $mean_abs_dev, $max_dev.
Examples
if (FALSE) { # \dontrun{
set.seed(110)
panel <- panel_from_margins(list(party = c(left = .5, right = .5)), n = 12)
instrument <- panel_instrument(item_choice("plan", "Which plan do you prefer?",
c("A", "B")))
cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b")
r <- panel_administer(panel, instrument, cfg)
r
bench <- data.frame(item_id = "plan", response = c("A", "B"),
share = c(.5, .5))
panel_benchmark(r, bench, "toy human study")
} # }