Extract token log-probabilities from a response

Token-level log-probabilities turn a classification into a measurement: the probability the model assigned to its own answer is a confidence score you can calibrate, threshold, or carry into downstream models as a soft label. Request them at config time (llm_config(..., logprobs = TRUE, top_logprobs = 5)); this helper then returns them tidily.

Usage

llm_logprobs(x)

Arguments

x

An llmr_response object (from call_llm() and friends), or a result frame from call_llm_par() (whose response list-column holds the response objects).

Value

For a single response: a tibble with one row per generated token: token (character), logprob (double), and top_logprobs (a list-column of data frames with the k most likely alternatives at that position, when requested). Returns a zero-row tibble when the response carries no logprobs. For a result frame: a list of such tibbles, one per row.

See also

llm_config(), tokens()

Examples

if (FALSE) { # \dontrun{
# Provider support varies; deepseek-chat and OpenAI expose logprobs,
# Anthropic does not, and several hosts reject the flag model by model.
cfg <- llm_config("deepseek", "deepseek-chat",
                  logprobs = TRUE, top_logprobs = 3, temperature = 0)
r <- call_llm(cfg, "Answer with one word: is water wet?")
llm_logprobs(r)

# Confidence of the first answer token:
exp(llm_logprobs(r)$logprob[1])
} # }