Skip to contents

Detect shared state across experiment cells

Source: R/leakage.R
check_state_leakage.Rd

check_state_leakage() inspects the cells of an agent_experiment() (or a plain list of convertible results) and reports whether any cell shares a live agent with another. A correct experiment builds its agents inside run_fn, so each cell gets fresh agents with distinct ids; an agent built once outside run_fn and reused leaks its memory and identity across cells, which silently couples conditions that should be independent.

Usage

check_state_leakage(x)

Arguments

x

An agent_experiment() result (the tibble with a result list-column), or a plain list whose elements are Agents or agent_run objects (each element treated as one cell).

Value

An object of class agent_leakage_report: a list with leaks (a tibble with columns cell_i, cell_j, agent_id, kind, evidence), clean (TRUE when no leaks were found), and n_cells.

Details

The check is read-only and conservative: every cell is converted through as_agent_run() inside a tryCatch(), and a cell whose result is not run-able (a number, a string, anything off the provenance path) is skipped rather than treated as evidence. Two kinds of leak are reported:

  • shared_agent_instance: one agent_id participates in two different cells. Distinct cells with a fresh agent each cannot collide on an id, so a shared id is direct evidence of one reused instance.

  • memory_bleed: a stronger signal on top of a shared id, raised when the later cell's agent memory already contains content hashed from the earlier cell, i.e. the agent literally remembers the prior condition.

See also

agent_experiment(), as_agent_run()

Examples

if (FALSE) { # \dontrun{
design <- data.frame(framing = c("gains", "losses"))

# leaks: one agent is built once and reused by every cell
shared <- agent("S", LLMR::llm_config("groq", "openai/gpt-oss-20b"))
bad <- agent_experiment(design, reps = 1, run_fn = function(cond, rep) {
  shared$chat(cond$framing); shared
})
check_state_leakage(bad)          # clean = FALSE

# clean: a fresh agent is built inside every cell
good <- agent_experiment(design, reps = 1, run_fn = function(cond, rep) {
  a <- agent("S", LLMR::llm_config("groq", "openai/gpt-oss-20b"))
  a$chat(cond$framing); a
})
check_state_leakage(good)         # clean = TRUE
} # }