Skip to contents

Administer an instrument to a panel

Source: R/administer.R
panel_administer.Rd

Creates one request for each combination of persona and item. The persona goes in the system message, the item and its options in the user message. A closed-item reply is matched against the options offered, and anything unmatched becomes NA. Open items come back verbatim.

Usage

panel_administer(
  panel,
  instrument,
  config,
  max_calls = 5000L,
  confirm = FALSE,
  price_table = NULL,
  tokens_per_call = NULL,
  .runner = NULL,
  ...
)

Arguments

panel

A panel_from_margins(), panel_from_data(), or panel_from_personas() result.

instrument

A panel_instrument().

config

An LLMR::llm_config() for a generative model.

max_calls

Integer. If the run would make more than this many calls (personas times items), it stops unless confirm = TRUE. Default 5000.

confirm

Logical. Set TRUE to proceed past max_calls.

price_table, tokens_per_call

Optional. When both are supplied, the preflight reports a cost figure computed from your own price_table (the LLMR::llm_usage() format: columns model, input, output, prices per million tokens) and your tokens_per_call assumption. One number is a per-call total, priced as a range from all-input to all-output; two, c(input, output), price exactly. The package itself ships no prices and estimates no token counts.

.runner

Optional runner for offline or deterministic testing: a function(experiments, ...) that receives a data frame with config and messages list-columns and returns those rows with request_id and response_text columns. Each submitted request_id must appear once; returned rows may be in any order. Defaults to a live LLM call via LLMR::call_llm_par().

...

Passed to the runner (e.g. tries, progress).

Value

A panel_responses object with fields data, panel, instrument, benchmark, and usage. data is a tibble with persona_id, item_id, type, item_position (the item's fixed 1-based position in the instrument; each request is independent, so no questionnaire order is ever shown to the model), option_order (what this respondent saw, |-separated), response (matched option or NA; verbatim text for open items), and score (1-based scale position for Likert items). score uses the item's canonical scale rather than its displayed order. response_text, response_id, success, error_message, finish_reason, model, and provider retain execution provenance as ordinary columns. Conjoint administrations also include a profiles list-column. benchmark is NULL until panel_benchmark() is called. usage retains execution diagnostics and any token counts or per-call duration; it is NULL when the runner returned none of those usage fields.

Examples

set.seed(110)   # the panel draw is local; the model call is not
panel <- panel_from_margins(list(party = c(left = .5, right = .5)), n = 6)
instrument <- panel_instrument(
  item_likert("wk4", "A four-day work week would benefit society."),
  randomize = character(0))
cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b")
if (FALSE) { # \dontrun{
resp <- panel_administer(panel, instrument, cfg)
resp
} # }

# The `.runner` seam answers without a provider, for tests or for a
# deterministic or external respondent:
deterministic <- function(experiments, ...) {
  experiments$response_text <- "agree"
  experiments
}
panel_administer(panel, instrument, cfg, .runner = deterministic)