The LLMR ecosystem

LLMR provides the common interface to language-model providers. The other packages implement agents, model-persona panels, group discussions, text measurement, robustness analysis, and replication archives.

Why this exists

Model-assisted research poses distinct design problems across research tasks. Provider-independent experiments need comparable configurations and call records. Governed agents need explicit personas, tools, memory, budgets, and coordination rules. Persona-panel instruments need respondent-level randomization and comparison with human benchmarks. Moderated discussions need stated participant construction, moderation, and turn-level experimental designs. Text measurement needs codebooks, held-out human labels, uncertainty estimates, and records that later readers can check. The family separates these tasks while sharing provider access and result conventions.

Research tasks

compareLLMR
provider-independent calls and experiments
governLLMRagent
agent roles, budgets, and coordination
simulate or pilotLLMRpanel · FocusGroup
instruments and moderated discussions
measureLLMRcontent
validated coding, robustness, and replication

Which package?

Package	Use it when	Not for
LLMR	Calling chat or embedding models through a common interface and comparing models or conditions	Defining a content measure, persona panel, governed agent, or moderated discussion
LLMRagent	Constructing governed agents and comparing agent designs	Persona-panel instruments or moderated group discussions
LLMRcontent	Validated text measurement, robustness across coding choices, and replication archives	Accessible qualitative coding or text segmentation
LLMRpanel	Survey or experiment design with panels of model personas	Human-population estimates without comparison against a human benchmark
FocusGroup	Simulating moderated group discussion to pilot instruments or probe how a turn shifts the next	Estimating quantities about real human populations

Each package reference site documents its research workflow, result objects, and scope.

The packages

LLMR

CRAN

LLMR provides one configuration and calling interface for hosted and local language models. The same functions serve chat and embedding models and extend to multi-model and multi-condition studies.

Site GitHub

LLMRagent

GitHub; CRAN submission imminent

LLMRagent constructs language-model agents with personas, tools, memory, and budgets. It coordinates agents through delegation, fixed pipelines, shared conversations, and factorial experiments.

Site GitHub

LLMRcontent

GitHub; CRAN submission imminent

LLMRcontent codes text from a specified codebook and evaluates the coding against held-out human labels. It compares estimates across measurement choices and builds replication archives from LLMR audit logs.

Site GitHub

LLMRpanel

GitHub; CRAN submission imminent

LLMRpanel administers survey and experimental instruments to panels of model personas. It supports Likert, choice, open-response, and conjoint instruments, respondent-level randomization, and comparison against a human benchmark.

Site GitHub

FocusGroup

GitHub; CRAN submission imminent

FocusGroup runs moderated discussions among language-model agents and supports experiments that vary one turn and measure the next response. Its Shiny interface (run_focus_studio()) runs sessions and supports transcript analysis.

Site GitHub

Point-and-click

Three of the packages ship a Shiny front end, so the same workflows run without writing code. The front ends read API keys from environment variables and include an offline demonstration mode.

Content analysis. LLMRcontent::run_content_studio() -- build a codebook, seal a gold set, run a coding tournament, and read the validation report.
Persona-panel surveys. LLMRpanel::run_panel_studio() -- draw a persona panel from margins, administer Likert and choice items, and read the benchmark banner.
Focus groups. FocusGroup::run_focus_studio() -- run a moderated session, analyze a transcript, or run a continuation experiment that perturbs one turn and compares the next.

All three are built on LLMR.shiny, the shared GUI infrastructure that ships with the GUI packages and is not used directly. Its components supply the provider sidebar, key handling, usage accounting, demo/live runners, and display adapters.

Reproducibility principles

Validation inputs. LLMRcontent uses codebooks, gold sets, and model comparisons in its measurement workflow.
Audit records. LLMR logs model calls, and LLMRcontent builds hash-sealed archives from those logs.
Result diagnostics. Results retain test-split evaluations, panel benchmark status, and fragility measures.
Local execution. Validation and audit methods can use local open-weight models. When local models are used, restricted text remains on the host machine.

Install

install.packages(c("LLMR", "LLMR.shiny"))  # CRAN

# the research packages not yet on CRAN, from GitHub:
remotes::install_github("asanaei/LLMRagent")
remotes::install_github("asanaei/LLMRcontent")
remotes::install_github("asanaei/LLMRpanel")
remotes::install_github("asanaei/FocusGroup")