Administer an instrument to a panel — panel

Creates one independent request for each persona-item pair. The persona is placed in the system message, and the item and its options are placed in the user message.

Usage

panel_administer(
  panel,
  instrument,
  config,
  max_calls = 5000L,
  confirm = FALSE,
  price_table = NULL,
  tokens_per_call = NULL,
  .runner = NULL,
  ...
)

Arguments

panel: A result from panel_from_margins(), panel_from_data(), or panel_from_personas().
instrument: A panel_instrument() object.
config: An LLMR::llm_config() for a generative model.
max_calls: Maximum planned request count before confirmation is required. The default is 5000.
confirm: Set TRUE to proceed when the request count exceeds max_calls.
price_table, tokens_per_call: Optional inputs for a preflight cost calculation. Supply both. price_table uses the LLMR::llm_usage() format, with model, input, and output columns and prices per million tokens. A single tokens_per_call value gives a total priced over the possible input-output allocation. Two values, c(input, output), specify the allocation. The package supplies neither prices nor estimated token counts.
.runner: Optional function (experiments, ...) for offline or external execution. It receives a data frame with config and messages list-columns and returns the experiment rows with request_id and response_text. Each submitted request identifier must occur once. Returned rows may be reordered. The default uses LLMR::call_llm_par().
...: Passed to the runner.

Value

A panel_responses object.

data contains one row per persona-item pair. Its identifiers are persona_id, item_id, and type.
item_position is the item's fixed, 1-based instrument position. option_order records displayed options separated by |.
response contains the matched option or open-item text. score gives the 1-based position on a Likert item's supplied scale.
Execution columns retain response_text, response_id, success, error_message, finish_reason, model, and provider. Request hashes, model versions, and creation times are retained when supplied by the runner.
Conjoint administrations include recorded profile draws in a profiles list-column.
panel and instrument retain the supplied study objects. benchmark is NULL until panel_benchmark() is called.
usage retains available token or duration records and their execution identifiers. It is NULL when no usage fields were returned.

Details

Closed-item replies are matched to the offered options. Unmatched replies receive NA. Open-item replies retain their text with surrounding whitespace removed.

Requests do not contain earlier items or responses. Instrument position therefore does not describe an order of exposure.

With "option_order" randomization, choice options are permuted for each request. Likert scales retain their supplied order or are reversed, each with probability one-half. Likert scores always use the supplied scale order.

Examples

set.seed(110)
panel <- panel_from_margins(
  list(party = c(left = .5, right = .5)), n = 6
)
instrument <- panel_instrument(
  item_likert("wk4", "A four-day work week would benefit society.")
)
cfg <- LLMR::llm_config("groq", "example-model")
runner <- function(experiments, ...) {
  experiments$response_text <- "agree"
  experiments
}
responses <- panel_administer(
  panel, instrument, cfg, .runner = runner
)
responses$data