Apply an LLM prompt over vectors/data frames

Usage

llm_fn(
  x,
  prompt,
  .config,
  .system_prompt = NULL,
  ...,
  .tags = NULL,
  .fields = NULL,
  .return = c("text", "columns", "object"),
  .na_action = c("send", "skip", "error"),
  .rows_per_prompt = 1L,
  .rowpack_payload = c("user", "system"),
  .rowpack_recovery = c("halve_recursive", "halve_once", "singletons", "retry_same",
    "none")
)

Arguments

x

A character vector or a data.frame/tibble.

prompt

A glue template string. With a data-frame you may reference columns ({col}); with a vector the placeholder is {x}.

.config

An llm_config object.

.system_prompt

Optional system message (character scalar).

...

Passed unchanged to call_llm_broadcast() (e.g. tries, progress, verbose).

.tags

Optional character vector of XML-like tag names to request and parse. When supplied, delegates to llm_fn_tags() for tag-based extraction.

.fields

Optional field selector for tag extraction (see llm_fn_tags()).

.return

One of c("text","columns","object"). "columns" returns a tibble of diagnostic columns; "text" returns a character vector; "object" returns a list of llmr_response (or NA on failure).

.na_action

What to do with elements whose template references an NA value. "send" (default) renders NA as an empty string and sends the prompt anyway; "skip" does not call the API for those elements and returns NA for them (finish_reason = "skipped" in column mode); "error" stops before any call is made. "skip"/"error" are not available together with .tags.

.rows_per_prompt

Integer scalar, or Inf. Number of input elements packed into a single generative request. The default, 1, sends one request per element (the historical behaviour). When greater than 1, elements are grouped and transmitted in one call wrapped in numbered <row_1>...</row_1> tags (see Row batching below); Inf sends all elements in a single call. Ignored for embedding configurations, which use get_batched_embeddings() instead.

.rowpack_payload

One of c("user","system"). Channel to which the <row_i> data block is appended when batching. The imperative instruction is always placed in the system message; this argument controls only where the data goes. The default, "user", keeps a static system prompt cacheable.

.rowpack_recovery

How to handle rows that a batched call leaves unresolved (dropped, malformed, or truncated). One of:

"halve_recursive": (default) re-issue the unresolved rows at half the batch size, recursing down to single rows.
"halve_once": re-issue at half the batch size exactly once, then give up on any still-unresolved rows.
"singletons": re-issue each unresolved row on its own.
"retry_same": re-issue the failed batch once at the same size.
"none": do not recover; unresolved rows are returned as NA.

Recovery is bounded by an internal call budget so it always terminates.

Value

For generative mode:

.return = "text": character vector
.return = "columns": tibble with diagnostics
.return = "object": list of llmr_response (or NA on failure; unavailable when .rows_per_prompt > 1) For embedding mode, always a numeric matrix.

Batching, chunking, and row packing

"Batch" appears in three distinct senses in LLMR, and they are easy to keep apart once you note which path each belongs to. (1) The asynchronous provider Batch API (llm_batch_submit() and friends) defers a whole job for later delivery at a reduced price; it is the only deferred path here. (2) In the embedding path, get_batched_embeddings()'s batch_size sets how many texts go in one synchronous embedding request, bounded by the provider's per-call limit. (3) In the generative path, the .rows_per_prompt argument here packs several data rows into one generative prompt and parses them back into rows. Senses (2) and (3) are synchronous: batch_size and .rows_per_prompt control how work is grouped into requests, not the per-token rate, so the synchronous helpers do not themselves apply a provider discount. Pricing is a separate axis: several providers bill batched embeddings at a reduced rate through a dedicated async/batch tier (the embedding analogue of (1)), so embeddings are not categorically full-price; that discount is a property of the endpoint, not of batch_size. Row packing (3) can still lower total tokens by amortizing shared prompt overhead across rows, again without changing the rate.

Row packing

With .rows_per_prompt > 1, several input elements travel in one generative request: LLMR wraps each element's prompt in a numbered tag, <row_1>...</row_1>, <row_2>...</row_2>, and so on, appends that block to the message (see .rowpack_payload), and instructs the model to answer each item inside a matching numbered tag. The reply is split back into the original elements by those numbers. Batching trades a smaller number of (larger) requests for some dependence on the model following the protocol; it is most useful with capable models at temperature = 0, and it is a net loss when the model ignores the wrapping. Results are deterministic given the model's outputs: partitioning and parsing add no randomness. Rows the model drops, reorders, duplicates, or truncates are detected and re-issued according to .rowpack_recovery. Because a batch shares one underlying call, token counts are reported once per batch (on its first resolved row, NA elsewhere), as is the wall-clock duration, so that summing those columns is correct. When a batch reply is entirely unusable and its rows succeed only through recovery calls, the failed call's spend has no successful row to land on, so sums can slightly undercount in heavy-recovery runs.

Examples

if (FALSE) { # \dontrun{
words <- c("excellent", "awful")
cfg <- llm_config("groq", "openai/gpt-oss-20b", temperature = 0)
llm_fn(words, "Classify '{x}' as Positive/Negative.", cfg, .return = "text")

df <- tibble::tibble(text = words, source = c("review", "review"))
llm_fn(df, "Classify '{text}' from {source}.", cfg, .return = "columns")
} # }