Skip to contents

Like call_llm(), but the reply arrives incrementally: callback is invoked with each text chunk as it is generated, and the complete llmr_response is returned at the end. Streaming keeps long generations responsive and avoids HTTP timeouts on slow, lengthy completions.

Usage

call_llm_stream(
  config,
  messages,
  callback = function(chunk) cat(chunk),
  verbose = FALSE
)

Arguments

config

An llm_config for a generative model.

messages

Messages as in call_llm().

callback

Function called with each text chunk (a character scalar) as it arrives. The default prints chunks to the console with cat(). Reasoning deltas (when a provider streams them separately) are not passed to callback; they are collected into the result's thinking field.

verbose

Print the assembled response object at the end.

Value

An llmr_response assembled from the stream (invisibly). Token usage is filled when the provider reports it in the stream; otherwise it is NA.

Details

Supported providers: all OpenAI-compatible chat APIs (openai, groq, together, deepseek, xai, alibaba, zhipu, moonshot, xiaomi, openrouter, meta, ollama), Anthropic, and Gemini. The request body is built by the same internals as call_llm(), so parameters and structured output behave identically; the request_modifier and req_builder hooks and the request timeout apply as well. Only the transport differs. The response_modifier hook is the one exception: it rewrites a full parsed response, which has no per-chunk meaning for a stream, so it is not applied here.

A stream counts as complete only when the provider signals completion. An error event sent inside the stream, or a connection that ends before the provider's completion signal, raises a typed llmr_api_*_error that also has class llmr_stream_error; the text received before the failure is in the condition's partial_text field. The call is not retried, because callback may already have emitted part of the text. An HTTP error status returned before any text arrives raises the same typed llmr_api_*_error as call_llm(), without the llmr_stream_error class. Events that cannot be parsed are skipped with a warning that gives their number. An error raised by callback itself propagates unchanged.

Examples

if (FALSE) { # \dontrun{
cfg <- llm_config("groq", "openai/gpt-oss-20b")
r <- call_llm_stream(cfg, "Tell a 100-word story about a lighthouse.")
tokens(r)
} # }