Run a Focus Group Simulation — run_focus

Constructs the agents, moderator guide, and turn-taking flow, then runs one focus group session. config has no default; every agent and group-level call uses the one supplied. Before a live run, the function estimates the number of generated model outputs and applies the max_calls limit unless confirm is TRUE.

Usage

run_focus_group(
  topic,
  config,
  n_participants = 6,
  guide = c(Opening = 1, Icebreaker = 1, Engagement = 2, Exploration = 3, Closing = 1),
  demographics = NULL,
  survey_responses = NULL,
  flow = "desire_based",
  seed = NULL,
  message_mode = c("roleflip", "flat"),
  verbose = TRUE,
  max_participant_responses = NULL,
  max_calls = 120L,
  confirm = FALSE,
  max_tokens_config = NULL,
  reasoning_effort = NULL,
  on_desire_failure = c("error", "random"),
  injections = NULL,
  probes = NULL,
  moderator_verbatim = FALSE,
  call_on_silent = c("opening", "midpoint", "end"),
  call_on_text = NULL,
  handles = NULL,
  final_summary = TRUE,
  .runner = NULL
)

Arguments

topic: Character. Focus group topic.
config: An explicit LLMR::llm_config used by all agents and group-level model tasks.
n_participants: Integer. Number of participants, excluding the moderator.
guide: A named numeric vector or named list. The names are phases, among "Opening", "Icebreaker", "Engagement", "Exploration", and "Closing", each at most once and in that order. Numeric values select that many moderator instructions from the phase banks. Character vectors supply the ordered instructions directly. The default is a compact session (one opening, one icebreaker, two engagement and three exploration questions, one closing); with the default flow, participants, response limit, and call-ons it plans at most 111 model outputs, inside the default max_calls of 120. Larger guides multiply quickly: each question round costs one output per response plus, under "desire_based", one scoring output per participant. NULL runs a minimal script: an opening, one open discussion of topic, and a closing.
demographics: Optional participant demographics, one row per participant. With this, survey_responses, and personas all absent, the participants are drawn from the bundled ANES 2024 records.
survey_responses: Optional participant survey answers, one row per participant.
flow: Character. One of "round_robin", "probabilistic", or "desire_based".
seed: Optional whole number (a fraction such as 1.7 is an error) governing in-package sampling: the respondents drawn from the bundled records and the random choices of the turn-taking flow, such as the speaker draws of "probabilistic" and the tie-breaks of "desire_based". The random waits of a provider's retries do not draw from this stream. The caller's random-number state is restored when the function returns. The option focusgroup.seed, when set, fixes the respondent draw instead.
message_mode: Character. "roleflip" or "flat".
verbose: Logical. Print session progress.
max_participant_responses: Optional integer maximum number of participant responses per moderator question. NULL, the default, uses the option focusgroup.max_participant_responses, which is 2 unless you set it.
max_calls: Integer. Maximum estimated live model outputs allowed without confirmation; a finite whole number.
confirm: Logical. Permit a live run whose estimate exceeds max_calls.
max_tokens_config: Optional named list of token limits by call type, as for FocusGroup$new(): utterance, moderator, desire, desire_retry, summary, and interim_summary. A model that reasons before it answers needs desire and desire_retry large enough to hold its reasoning and the score.
reasoning_effort: Optional reasoning-effort setting passed to the provider: one string, such as "low", for every call, or a named vector by call type ("moderator", "participant", "desire", "summary", "probe"). It is written over the reasoning_effort of config for those calls; NULL leaves config as it is. LLMR forwards it on OpenAI-compatible providers.
on_desire_failure: Character. Under "desire_based", what happens when a participant's desire score cannot be obtained after its retry. "error" (the default) stops the session with an error of class focusgroup_desire_failure, returned as a partial session as described in Details. "random" chooses the speaker among the participants with valid scores by the usual rule, and uniformly among the eligible participants only when none has a valid score; the failure is recorded in the selected message's metadata$desire_scoring_failure.
injections: Optional fg_injection() or list of injections, each placed into the session at its point. To compare sessions that share everything before an injection, use run_focus_group_arms().
probes: Optional fg_probe() or list of probes, private questions asked of each participant at stated points. The answers are returned in probes and never enter the discussion.
moderator_verbatim: FALSE (the default) has the moderator's model render each guide item as a line of its own. TRUE delivers every guide item exactly as written, with no model call, and a vector of item numbers delivers those items so. A verbatim message carries metadata$verbatim = TRUE. The opening and closing instructions of the numeric phase banks are instructions to the moderator, so verbatim delivery suits a guide written as the lines to be said.
call_on_silent: Where the moderator calls on the participants who have not spoken since the previous such point, as described in Details: any of "opening", "midpoint", and "end" (the default is all three), guide item numbers, or "none" (or FALSE) for no call-ons.
call_on_text: The call-on line. It must contain {participant}, which is replaced by the participant's handle; NULL uses "{participant}, I'd like to hear from you as well. What are your thoughts?".
handles: The labels under which the agents see each other: NULL (the default) for "P1", "P2", ... and "Moderator", "ids" for the agent IDs, or a character vector of handles named by agent ID, as for FocusGroup$new().
final_summary: Logical. TRUE (the default) ends the session with a model-written summary; FALSE makes no summary call and leaves summary empty.
.runner: Optional function used instead of live model calls. It receives a data frame with config and messages list-columns and returns those rows with at least response_text. Token columns such as sent_tokens, rec_tokens, and total_tokens are optional; when they are absent, usage is unknown.

Value

A focus_group_result, a list with these components:

focus_group: The FocusGroup object that ran the session. It keeps the model configuration, so model analyses can be run on it later.
transcript: A data frame with one row per logged message, starting with the "System" message that lists the roster. Its columns are message_id, round, item (the guide item the message belongs to, 0 for the roster row), speaker_id (the agent ID), handle (the label the agents saw), is_moderator, text, phase, response_id, finish_reason, sent_tokens, rec_tokens, total_tokens, duration_s, provider, model, timestamp (in UTC), and the list-column metadata, which marks, among others, delivered lines (verbatim), injected messages (injected, injection), call-on lines (call_on, a list of point, participant, and handle), the replies to them (called_on), replies to a moderator line that addressed the speaker (addressed), replies whose self-label was removed (self_label_removed), replies whose opening self-introduction was removed (preamble_removed), and injected participant turns (as_turn; see fg_injection()).
summary: The final summary. It is NA when the final summary call failed, and character(0) in a partial result that stopped before it or in a session run with final_summary = FALSE.
participants: A data frame with one row per agent: id, handle, role, is_moderator, persona, communication_style (the agent's style instruction), demographics, survey_responses, provider, and model. When the participants were drawn from the bundled ANES records, the rows drawn are recorded in attr(result$focus_group$agents, "source_rows").
usage: A list of the token subtotals sent, rec, and total, and the count unknown described in Details. It records the session as run; model analyses run later add to the counters of focus_group but not to this list.
metadata: A list with topic, purpose, flow, message_mode, n_participants, estimated_calls, provider, model, complete (TRUE for a finished session, FALSE for a partial one), and failure (NULL, or a list with the phase and message of the error that stopped the session); then the conditions of the run: max_participant_responses, seed (NA when none was given), on_desire_failure, reasoning_effort, max_tokens (the token limit of each call type), call_settings (a data frame giving, for each call type and agent, the provider, model, and model parameters the calls start from), prompt_templates, standing_rules (the system-message rules for participants and the moderator), disclosure, versions (of FocusGroup, LLMR, and R), started, and branch (NULL, or for a session run by run_focus_group_arms() its arm and branch point), handles (a data frame of agent_id and handle), call_on_silent (a list of the points and the guide items they fall on, both empty when call-ons are off), call_on_text, final_summary, guide (the guide items as the session ran them, each a list with phase and, when set, text and verbatim), and flow_params (the parameters of the turn-taking rule: initial_propensities and recovery_increment for "probabilistic", min_desire_threshold and on_desire_failure for "desire_based", none otherwise). It holds no configuration object.
calls: A data frame with one row per model call the session made, in order, including retries and failed calls: call_id, call_type ("moderator", "participant", "desire", "summary", "interim_summary", or "probe"), agent_id, attempt (2 for a retry), round, phase, message_id (the message the call produced, if any), provider, model, the list-columns settings (the model parameters sent) and messages (the messages sent), raw_text (the reply before any cleaning), thinking (reasoning the provider returned separately), cleaned_text, score (for desire calls), finish_reason, response_id, sent_tokens, rec_tokens, total_tokens, reasoning_tokens, duration_s, error (the error a failed call raised), timestamp, and arm (the arm of a session continued by run_focus_group_arms(), otherwise NA). Only scalar sampling and length parameters enter settings; keys and endpoint addresses never do.
routing: A data frame with one row per speaker selection: pick, round, phase, turn (the participant turn it chose for), flow, rule (such as "rotation", "propensity_draw", "highest_score", "threshold_lowered", "highest_below_threshold", "uniform_after_failure", or, for turns the flow did not choose, "called_on", "addressed", and "injected"), selected_id, message_id, trigger_message_id (for "called_on" and "addressed", the moderator line that gave the turn; otherwise NA), threshold, tie_break ("none", "random", or "draw"), scoring_mode, and the list-columns candidates, excluded (named by participant, giving the reason, such as "previous_speaker" or "score_failed"), scores (desire scores, or selection probabilities for "probabilistic"), and tied, followed by arm.
probes: A data frame with one row per probe answer: probe (its label), point, agent_id, question, answer (the final answer, or NA when none came), value (the number read from the answer for a probe with a scale, otherwise NA; see fg_probe()), raw_text, finish_reason, attempts (the calls made for the answer, retries and the re-ask included), call_id (the row in calls of the call that gave the answer), round, phase, turn (the next participant turn), after_message_id (the last message before it), and arm.
injections: A data frame with one row per injection: label, arm, point, delivered (FALSE for one whose point the session never reached, which is also reported with a warning), message_id, speaker_id, from, visibility, text, cut_after_message_id (the message it follows), round, phase, turn, as_turn (TRUE for a participant's injected turn), and replaced (for such a turn, the turn it took the place of: "flow", "addressed", or "called_on"; "none" for a line from someone other than the participant called on, who still answers next).

Results saved by FocusGroup 0.5.2 and earlier lack usage$unknown, metadata$complete, and metadata$failure; results saved by 0.5.3 and earlier lack calls, routing, probes, injections, and the metadata that follows failure; results saved by 0.5.4 and earlier lack the item and handle columns, trigger_message_id, and the metadata that follows branch; results saved by 0.5.5 and earlier lack the value and attempts columns of probes, communication_style in participants, as_turn and replaced in injections, and metadata$guide and metadata$flow_params.

A result is saved whole with saveRDS(), and its focus_group then holds the model configuration, which may contain a literal API key or a private endpoint, and the runner. format() writes the record without them, one JSON object per line, for a file that focus_group_from_transcript() reads back; see format.focus_group_result().

Details

At the end of each call-on item's discussion the moderator calls on every participant who has not spoken since the previous call-on point, or since the start for the first, in roster order. "opening" is the first guide item participants answer, "midpoint" the middle one, and "end" the last; points that fall on the same item count once, and item numbers count every guide item, as fg_after_item() does. Each call-on line is delivered as written, with no model call, and carries metadata$call_on; the participant then answers, with a cue in the turn instruction and no desire score, and the reply carries metadata$called_on = TRUE. With the default guide the points are items 2, 4, and 7, so every participant speaks during the icebreaker. Krueger (2002) describes the opening question of a focus group as a round robin.

The participants and the moderator see each other under handles, never under the IDs of the records: the IDs of create_agents() are already "P1", "P2", ..., and the moderator appears as "Moderator". When the moderator's last question opens or closes with one participant's handle ("P3, do you agree?", "P3: what do you think?", "Could you say more, P3?"), or is a short follow-up naming one ("And P3?", "What about P3?"), and does not also put it to the group, that participant takes the next turn, without a desire score and with a sentence in its turn instruction saying that the question was put to it, and the reply carries metadata$addressed = TRUE. A label a speaker writes for itself at the start of its reply (**MOD:**, P3:) is removed, and the message carries metadata$self_label_removed = TRUE. A short opening self-introduction ("As a teacher, ", "As a retired nurse, ") is removed too, and the message carries metadata$preamble_removed = TRUE; a longer opening clause, or one that belongs to the statement ("As a teacher I grade papers at night, ...", "As a result, ...", "As an alternative, ..."), is kept. In both cases the reply as returned stays in calls.

The estimate counts one output for each moderator turn the model writes, one for each participant response a question round allows, one for each participant at each call-on point, one for each probe answer (two when the probe has a scale, for a possible re-ask), and one for the final summary. Under "desire_based" it adds one desire score per participant before each response the flow chooses. With the default guide, six participants, and the default flow, two responses per question give an estimate of 111 outputs (93 without call-ons) and three give 153. A session can make fewer calls, since the desire-based flow does not score the previous speaker, an addressed turn needs no scores, and a call-on point calls on only the participants who have been silent, or more, since incomplete replies are retried and a long discussion triggers interim summaries. max_calls applies to the estimate, and only to live runs: with .runner the session runs whatever the estimate.

When a model call fails partway through the session (a provider error that persists after retries, an error from .runner, a reply still incomplete after its retry, or a failed interim summary), the turns already completed are not lost. The function raises an error of class focusgroup_incomplete_session whose partial field holds the focus_group_result built from those turns, with metadata$complete set to FALSE. The error's phase field names the script phase that was running, such as "engagement_question", or "interim_summary", and its parent field holds the original error. Errors raised before the session starts, such as an invalid argument or the max_calls check, carry no partial result. A failed final summary is not an error of this kind: the function warns and returns the complete transcript with an NA summary.

Token usage that a provider did not report is unknown, not zero. A call's total is its reported total, or the sum of its sent and received counts when both are known. The usage component adds the reported counts of every call FocusGroup makes, including retries of incomplete replies, desire scores, and interim summaries, and its unknown element counts the calls whose usage was incomplete. When unknown is positive the subtotals are lower bounds, and when it is NA, as after a call that raised an error, the number of such calls cannot be determined. Retries made inside LLMR::call_llm_robust() are not visible here and are not counted. A reported count that is not a nonnegative whole number is treated as unknown, with a warning.

In the transcript, a message's token count is NA unless every call behind the message reported it. The message's metadata element usage_unknown gives the number of those calls whose usage was incomplete.

References

Krueger, R. A. (2002). Designing and Conducting Focus Group Interviews. University of Minnesota.

Examples

cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b",
                        api_key = LLMR::llm_api_key_env("GROQ_API_KEY"))
guide <- list(
  Opening = "Welcome the participants and state the ground rules.",
  Exploration = "Which funding priority deserves attention first?",
  Closing = "Thank the participants and close the session."
)

scripted <- function(experiments, ...) {
  experiments$response_text <- "Evening opening hours matter most to me."
  experiments
}
result <- run_focus_group(
  topic = "Library funding priorities",
  config = cfg,
  n_participants = 2,
  guide = guide,
  flow = "round_robin",
  verbose = FALSE,
  .runner = scripted
)
result$transcript[, c("speaker_id", "phase", "text")]
#>   speaker_id                phase                                     text
#> 1     System                setup     Participants: P1, P2; Moderator: MOD
#> 2        MOD              opening Evening opening hours matter most to me.
#> 3        MOD exploration_question Evening opening hours matter most to me.
#> 4         P1 exploration_question Evening opening hours matter most to me.
#> 5         P2 exploration_question Evening opening hours matter most to me.
#> 6        MOD              closing Evening opening hours matter most to me.

# The scripted runner reports no token counts, so usage is unknown.
result$usage
#> $sent
#> [1] 0
#> 
#> $rec
#> [1] 0
#> 
#> $total
#> [1] 0
#> 
#> $unknown
#> [1] 6
#> 

# A session that fails partway returns what it completed.
failing <- local({
  calls <- 0
  function(experiments, ...) {
    calls <<- calls + 1
    if (calls == 3) stop("The provider is unavailable.")
    scripted(experiments)
  }
})
partial <- tryCatch(
  run_focus_group("Library funding priorities", cfg, n_participants = 2,
                  guide = guide, flow = "round_robin", verbose = FALSE,
                  .runner = failing),
  focusgroup_incomplete_session = function(e) e$partial
)
partial$metadata[c("complete", "failure")]
#> $complete
#> [1] FALSE
#> 
#> $failure
#> $failure$phase
#> [1] "exploration_question"
#> 
#> $failure$message
#> [1] "The provider is unavailable."
#> 
#>