Summarize a Discussion by Participant, Item, Disagreement, Trajectory, Turn, or Session

Summarizes a finished or imported discussion with a model, in one of six units and at one or more lengths. The model is asked for claims as JSON: each claim states one point, its status, the messages that support it, and any words quoted from them. Every claim is checked against the transcript before it is used, and the prose of the summary is rendered from the claims that pass. No message is added to the session and no agent sees the summary.

Usage

summarize_focus_group(
  x,
  config,
  by = "participant",
  compression = "brief",
  include_probes = FALSE,
  on_fail = c("error", "drop"),
  max_calls = 30L,
  confirm = FALSE,
  max_tokens = NULL,
  reasoning_effort = NULL,
  templates = NULL,
  verbose = FALSE,
  .runner = NULL
)

Arguments

x: A focus_group_result from run_focus_group(), a focus_group_arms object from run_focus_group_arms(), a FocusGroup, including one made by focus_group_from_transcript(), or the list that FocusGroup$run_arms() returns (or its named list of arms), summarized as a focus_group_arms object is.
config: An explicit llm_config object for the summarizing model. The session's own models and runner are never used.
by: The unit: one of "participant", "item", "disagreements", "trajectory", "turn", or "session".
compression: One or more of "headline", "brief", and "detailed".
include_probes: Logical. Send private probe answers to the model as a separate, labeled block.
on_fail: "error" or "drop"; see Checks.
max_calls: Maximum estimated model calls allowed without confirm.
confirm: Logical. Permit a run whose estimate exceeds max_calls.
max_tokens: Optional token limit for each reply. By default, the configuration's own limit, or else 1000, 2000, or 3000 tokens for the longest length asked for, with 120 more per message under by = "turn" and 300 more per stage under by = "trajectory".
reasoning_effort: Optional reasoning effort for every summary call, one string, written over the configuration's setting.
templates: Optional named list of templates replacing the defaults of the same name: summary_participant, summary_item, summary_disagreements, summary_trajectory, summary_turn, summary_session, and summary_reask (see get_default_prompt_templates()). A unit template must contain \{\{transcript\}\} and \{\{format\}\}, and summary_reask must contain \{\{failures\}\}; a placeholder without a value is an error.
verbose: Logical. Print one line per call.
.runner: Optional function used instead of live model calls. It receives a data frame with config and messages list-columns and returns the rows with at least response_text.

Value

A focus_group_summary, a list with:

summaries: One row per unit and length: segment ("session", or "prefix" and the arm names), unit, level, text (rendered from the claims kept; NA when there is none), n_claims, words, budget, n_turns, speaker_id (the user ID, for participant and turn units), and note.
claims: The claims kept, one row each: segment, unit, claim, kind, status, text, speakers (handles as the model saw them), speaker_ids, evidence, context, probe_evidence, act, ending, ending_message_id, levels, and attempt.
quotes: The quotes of the claims kept, each cut from its source: text as the model gave it, verbatim as it appears in the message, and start and end, character positions in the message text, with spans giving each part of a quote joined by "...".
checks: Every check run, one row per check, claim, and attempt, with passed and, on failure, detail.
rejected: Claims dropped under on_fail = "drop", with the reasons.
calls: One row per model call, in the format of the calls component of run_focus_group(), with segment and unit.
turns: The table from fg_turns(), with label, the handle the model saw.
metadata: by, compression, include_probes, on_fail, provider, model, settings, max_tokens, reasoning_effort, structured_output, templates, handles (user ID, handle, and the label sent to the model), segments, source_hash, units, estimated_calls, max_calls, budgets, usage, versions, created, complete, and failure.

as.data.frame() returns summaries, or another table named by its component argument.

Units

"participant": One call per participant: positions and reasons, changes of position, reactions to others, and guide items not addressed. A participant with no turns gets a row with n_turns = 0 and no call.
"item": One call per guide item with participant turns, sending only that item's messages: positions and their holders, agreement and disagreement. Who spoke, and how often, is counted in code and given to the model.
"disagreements": One call per session: one entry per disagreement or position held by one participant alone, with its holders, its messages, and how it ended (a concession, the moderator moving on, or unresolved).
"trajectory": One call per session. The stages are fixed by the package: thirds of the participant turns at "headline", one stage per guide item at "brief" and "detailed".
"turn": One call per guide item (an item without participant turns joins the next one): a gist of every message, with an act code at "detailed".
"session": One call per session for the whole group.

For a focus_group_arms object the shared prefix is summarized once and each arm from the branch point on; an arm's call sees only the messages after the branch.

Lengths

compression names the lengths: "headline" (one sentence per unit, 20 words per turn), "brief" (a paragraph of about 100 to 120 words; 40 words per turn), and "detailed" (longer, with quotes). All the lengths asked for come back in one call per unit, as lists of claims drawn from one set, so they agree with each other. The word budgets are recorded beside each summary; they are not enforced.

What the model sees

The transcript, with each message's id and its speaker's handle, the label the agents saw. No user ID labels a speaker: a speaker whose handle is its user ID (an imported transcript, handles = "ids", or a session made before handles existed) is shown as P1, P2, ... in roster order, unless that ID already has this form or another voice uses it, and the moderator as Moderator. The messages themselves are sent as recorded: the speakers wrote them under the handles they saw, and a name in them ("Ana, why evenings?", "Will the library stay open in May?") is left as written. Personas are never sent, nor is the System roster row. Injected messages are marked as injected, and those shown to participants only are marked as such. Private probe answers are sent only with include_probes = TRUE, as a separate block with ids Q1, Q2, ...

Checks

Each reply is checked in code, with no model involved. The JSON must parse (the reply is the first complete JSON object with a claims field, in a code block or in the text; only the final answer counts, so reasoning a model writes into its reply is ignored) and the reply must not have been cut off at the token limit. Every message a claim cites must exist, lie within the unit, and not be the System row. The messages given as evidence must be by the speakers the claim names, and a claim other than "not_stated" must cite at least one message by each speaker it names.

A quote must appear in the text of the cited message, ignoring only differences of case, whitespace, and quotation marks (and, for Persian, the zero-width non-joiner and the Arabic forms of yeh and kaf). It must have at least 3 words and start and end on word boundaries, an apostrophe or hyphen between letters counting as part of a word. The parts of a quote joined by "..." must appear in order in the same message, each with at least 2 words, and the words left out between them may not include a negation such as "not" or "n't". Words a claim puts in double quotation marks in its text must likewise appear in a message it cites as evidence.

A claim of change must cite two messages of one speaker, earlier first. Every handle of the form P and a number in a claim must be on the roster; a participant the text names must be among the claim's speakers or the speaker of a message it cites as context; any speaker the text names, by the label shown to the model or, for an imported transcript, by the speaker's name (matched in its case, as a whole word), must be among the claim's speakers or the speakers of the messages it cites, unless the name with a word beside it repeats a cited message ("open in May" from a message asking about May), so that "Ben said ..." citing only Ana's message fails; a claim that is not "not_stated" may not name a participant with no message in the unit; a claim that a participant said nothing fails when that participant spoke; and a user ID or name the model was not shown, as a speaker's label or in the text of a message, may not appear. A claim that speaks for the whole group ("everyone", "all of them", "no one", "consensus", "unanimous", "the whole group", and similar wording) must cite a message from every participant who spoke on each item it cites. Unless include_probes = TRUE, no claim may share a run of 6 words with a probe answer that does not also occur in the transcript.

Coverage is counted over the claims that pass: every participant, every participant who spoke on the item, every message, or every stage, as the unit requires, and every length asked for. A participant counts as covered only by a claim that reports what they said, with a message of theirs as evidence. The prose shows the status of a claim that is not "stated" at every length.

A reply that fails is sent back once, with each failure stated (for instance, that a quote is not in message 14, followed by the text of message 14); after a reply cut off at the token limit, the limit is doubled for the re-ask. If the second reply fails too, on_fail = "error" raises an error of class focusgroup_summary_check_failure with fields checks (the checks of both replies), replies (both raw replies), segment, unit, and partial (the summary of the units completed before it). on_fail = "drop" instead moves the failing claims to rejected and renders the summary from the rest, but still raises the error when a dropped claim reports a disagreement, a position held by one participant alone, or a change, or when the reply as a whole fails (invalid JSON, a cut-off reply, missing lengths, or coverage that the claims kept do not give).

Calls and structured output

Calls are made one at a time, with call types "summary_<by>", and are recorded in the calls component of the result, never in the session's records; the random-number state is the same after the call as before. The estimate checked against max_calls counts one re-ask per unit, an upper bound. A unit whose prompt is longer than the prompt window (64,000 estimated tokens, counting bytes for text outside ASCII) stops the run before any call, with an error of class focusgroup_summary_input_too_long; by = "item" sends one guide item at a time. The token limit is written into the field the configuration uses (max_completion_tokens, max_output_tokens, or max_tokens). When the installed LLMR provides LLMR::enable_structured_output(), the request also carries the JSON schema of the reply, unless the configuration already sets a response format or is an Anthropic configuration with thinking enabled, which cannot take a forced tool call; set options(focusgroup.summary_structured_output = FALSE) for a model that rejects a schema. Otherwise the format is stated in the prompt alone. Either way every reply is validated locally. The schema and the mode are recorded in metadata$structured_output. No temperature or seed is set.

Examples

transcript <- data.frame(
  speaker = c("Moderator", "Ana", "Ben"),
  text = c("What should change about the library hours?",
           "Evening hours would help working parents like me.",
           "I disagree, mornings are what retirees need most.")
)
fg <- focus_group_from_transcript(transcript, topic = "Library hours")
# A scripted model reply for each participant, as a live model would give.
scripted <- function(experiments, ...) {
  prompt <- experiments$messages[[1]][[1]]$content
  unit <- if (grepl("participant, P1", prompt, fixed = TRUE)) "P1" else "P2"
  claim <- if (unit == "P1") {
    list(id = 1, unit = "P1", kind = "position", status = "stated",
         text = "P1 wants evening hours for working parents.",
         speakers = list("P1"), evidence = list(2), context = list(1),
         quotes = list(list(message_id = 2, speaker = "P1",
                            text = "Evening hours would help")))
  } else {
    list(id = 1, unit = "P2", kind = "position", status = "stated",
         text = "P2 disagrees and puts retirees' mornings first.",
         speakers = list("P2"), evidence = list(3), context = list(2),
         quotes = list(list(message_id = 3, speaker = "P2",
                            text = "mornings are what retirees need")))
  }
  experiments$response_text <- as.character(jsonlite::toJSON(
    list(claims = list(claim), levels = list(headline = list(1))),
    auto_unbox = TRUE))
  experiments
}
cfg <- LLMR::llm_config("openai", "gpt-4o-mini")
out <- summarize_focus_group(fg, cfg, compression = "headline",
                             .runner = scripted)
out
#> <focus_group_summary>
#> By: participant | Lengths: headline
#> Model: openai/gpt-4o-mini | Calls: 2 | Structured output: schema
#> Claims kept: 2 | Quotes: 2
#> P1: P1 wants evening hours for working parents.
#> P2: P2 disagrees and puts retirees' mornings first.
out$quotes[, c("unit", "message_id", "verbatim")]
#>   unit message_id                        verbatim
#> 1   P1          2        Evening hours would help
#> 2   P2          3 mornings are what retirees need