Stream a chat completion token by token

Like call_llm(), but the reply arrives incrementally: callback is invoked with each text chunk as it is generated, and the complete llmr_response is returned at the end. Streaming keeps long generations responsive and avoids HTTP timeouts on slow, lengthy completions.

Usage

call_llm_stream(
  config,
  messages,
  callback = function(chunk) cat(chunk),
  verbose = FALSE
)

Arguments

config

An llm_config for a generative model.

messages

Messages as in call_llm().

callback

Function called with each text chunk (a character scalar) as it arrives. The default prints chunks to the console with cat(). Reasoning deltas (when a provider streams them separately) are not passed to callback; they are collected into the result's thinking field.

verbose

Print the assembled response object at the end.

Value

An llmr_response assembled from the stream (invisibly). Token usage is filled when the provider reports it in the stream; otherwise it is NA.

Details

Supported providers: all OpenAI-compatible chat APIs (openai, groq, together, deepseek, xai, alibaba, zhipu, moonshot, xiaomi, ollama), Anthropic, and Gemini. The request body is built by the same internals as call_llm(), so parameters and structured output behave identically; the request_modifier and req_builder hooks and the request timeout apply as well. Only the transport differs. The response_modifier hook is the one exception: it rewrites a full parsed response, which has no per-chunk meaning for a stream, so it is not applied here.

See also

call_llm(), chat_session()

Examples

if (FALSE) { # \dontrun{
cfg <- llm_config("groq", "openai/gpt-oss-20b")
r <- call_llm_stream(cfg, "Tell a 100-word story about a lighthouse.")
tokens(r)
} # }