Run a factorial agent experiment

Takes a design frame (one row per condition), runs run_fn once per condition and replication, and returns the design with rep, result (list-column), error, and duration columns. A failing cell records its error message and does not stop the others; re-run failures by filtering on !is.na(error).

Usage

agent_experiment(design, run_fn, reps = 1L, parallel = FALSE, quiet = FALSE)

Arguments

design

A data frame; each row is a condition, each column a factor of the design (personas, prompts, treatments, model names, ...).

run_fn

function(cond, rep) where cond is one row of design as a named list. Whatever it returns is stored in the result list-column.

reps

Replications per condition (default 1).

parallel

If TRUE, cells run concurrently via the future framework (set a plan first, e.g. future::plan(future::multisession)).

quiet

FALSE prints one progress line per completed cell.

Value

design expanded by replication, plus rep, result, error, and duration columns.

Details

run_fn receives the condition as a named list plus the replication number, and should build its agents inside the function so every cell starts fresh:

run_fn <- function(cond, rep) {
  cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b", temperature = 0.9)
  panel <- list(
    agent("A", cfg, persona = cond$persona_a),
    agent("B", cfg, persona = cond$persona_b)
  )
  deliberate(panel, cond$proposal, quiet = TRUE)
}

A note on seeds: model replies are sampled server-side, so a local seed never affects them. Set one only when your code draws random numbers itself (a run_fn that randomizes stimuli, or the "random" turn policy inside it); under parallel = TRUE the workers then use statistically sound parallel streams (future.seed). Combine with LLMR::llm_log_enable() to keep a per-call audit file of the whole experiment.

See also

deliberate(), debate(), LLMR::llm_usage()

Examples

if (FALSE) { # \dontrun{
design <- expand.grid(
  temperature = c(0.2, 1.0),
  framing = c("gains", "losses"),
  stringsAsFactors = FALSE
)
res <- agent_experiment(design, reps = 3, run_fn = function(cond, rep) {
  cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b",
                          temperature = cond$temperature)
  a <- agent("Subject", cfg, quiet = TRUE)
  a$reply(paste("Decide under", cond$framing, "framing: ..."))
})
} # }