Skip to contents

Record every LLM call in a local audit log

Source: R/audit_log.R
llm_log_enable.Rd

llm_log_enable() turns on a session-wide audit log: each API call made through LLMR (including those issued by llm_fn(), llm_mutate(), call_llm_par(), and chat_session()) appends one JSON object to path. llm_log_disable() turns logging off. llm_log_status() reports the current destination, if any.

Usage

llm_log_enable(path = "llmr_log.jsonl", include_messages = TRUE)

llm_log_disable()

llm_log_status()

llm_log_active()

Arguments

path

File path for the log. It is opened for append when logging is enabled, creating the file if needed; an unusable destination errors before logging is activated.

include_messages

Logical. If TRUE (default), the request body and the reply text are stored. If FALSE, only metadata is stored.

Value

llm_log_enable() and llm_log_disable() return the previous log path invisibly. llm_log_status() returns the active path or NULL, invisibly, after printing a one-line status.

llm_log_active() returns a list with active (logical), path (the destination or NULL), and include_messages (logical), for code that needs to read the logging state without printing.

Details

Methodological guidance for LLM-assisted research asks authors to retain, for every call: the model and provider, the full prompt, the inference settings, the output, and identifiers that allow an exact lookup later. The audit log records precisely that:

  • ts: ISO-8601 timestamp with timezone.

  • schema_version: the version of this record layout (currently "1.0"), so downstream tools that parse the log can rely on a stable contract.

  • provider, model: as configured; model_version: the identifier the server reports having served (when echoed), which catches silent model updates.

  • request: the JSON body sent to the provider, including all sampling parameters and the rendered messages. Inline file data (base64) is replaced by a short placeholder so logs stay small.

  • text, finish_reason, usage: the reply, why it stopped, and token counts (including cached tokens when reported).

  • response_id, status, duration_s: provider request id, HTTP status, and wall-clock seconds.

  • Failed calls are logged too (kind = "error"), with the provider's error message.

Records are appended line by line. Under multisession parallel execution, workers write per-process shard files, which LLMR folds back into the main log after the parallel call completes. The log contains your prompts and the model's replies in clear text. It never contains API keys.

Set include_messages = FALSE to omit request bodies and reply text (keeping only metadata, parameters, usage, and identifiers), e.g. when prompts contain confidential data.

See also

llm_usage() for token summaries, report() for a draft methods paragraph.

Examples

if (FALSE) { # \dontrun{
llm_log_enable("annotation_run.jsonl")
cfg <- llm_config("groq", "openai/gpt-oss-20b")
call_llm(cfg, "One word: capital of France?")
llm_log_disable()

# Read the log back as a data frame
log_df <- jsonlite::stream_in(file("annotation_run.jsonl"), verbose = FALSE)
} # }