Like call_llm(), but the reply arrives incrementally: callback is
invoked with each text chunk as it is generated, and the complete
llmr_response is returned at the end. Streaming keeps long generations
responsive and avoids HTTP timeouts on slow, lengthy completions.
Usage
call_llm_stream(
config,
messages,
callback = function(chunk) cat(chunk),
verbose = FALSE
)Arguments
- config
An llm_config for a generative model.
- messages
Messages as in
call_llm().- callback
Function called with each text chunk (a character scalar) as it arrives. The default prints chunks to the console with
cat(). Reasoning deltas (when a provider streams them separately) are not passed tocallback; they are collected into the result'sthinkingfield.- verbose
Print the assembled response object at the end.
Value
An llmr_response assembled from the stream (invisibly). Token
usage is filled when the provider reports it in the stream; otherwise it
is NA.
Details
Supported providers: all OpenAI-compatible chat APIs (openai, groq,
together, deepseek, xai, alibaba, zhipu, moonshot, xiaomi, openrouter, meta,
ollama), Anthropic, and Gemini. The request body is built by the same
internals as call_llm(), so parameters and structured output behave
identically; the request_modifier and req_builder hooks and the request
timeout apply as well. Only the transport differs. The response_modifier
hook is the one exception: it rewrites a full parsed response, which has no
per-chunk meaning for a stream, so it is not applied here.
A stream counts as complete only when the provider signals completion. An
error event sent inside the stream, or a connection that ends before the
provider's completion signal, raises a typed llmr_api_*_error that also
has class llmr_stream_error; the text received before the failure is in
the condition's partial_text field. The call is not retried, because
callback may already have emitted part of the text. An HTTP error status
returned before any text arrives raises the same typed llmr_api_*_error
as call_llm(), without the llmr_stream_error class. Events that
cannot be parsed are skipped with a warning that gives their number. An error
raised by callback itself propagates unchanged.
Examples
if (FALSE) { # \dontrun{
cfg <- llm_config("groq", "openai/gpt-oss-20b")
r <- call_llm_stream(cfg, "Tell a 100-word story about a lighthouse.")
tokens(r)
} # }