Skip to contents

Compares different configurations (models, providers, settings) using the same message. Perfect for benchmarking across different models or providers. Use setup_llm_parallel() when you want explicit control over workers.

Usage

call_llm_compare(configs_list, messages, ...)

Arguments

configs_list

A list of llm_config objects to compare.

messages

A character vector or a list of message objects (same for all configs).

...

Additional arguments passed to call_llm_par (e.g., tries, verbose, progress).

Value

A tibble with columns: config_index (metadata), provider, model, all varying model parameters, response_text, raw_response_json, success, error_message.

Parallel Workflow

Recommended workflow:

  1. Call setup_llm_parallel() once at the start of your script.

  2. Run one or more parallel experiments (e.g., call_llm_broadcast()).

  3. Call reset_llm_parallel() at the end to restore sequential processing. If the active future plan is sequential, call_llm_par() temporarily switches to multisession for the duration of the call.

Examples

if (FALSE) { # \dontrun{
  # Compare different models
  config1 <- llm_config(provider = "openai", model = "gpt-5-nano")
  config2 <- llm_config(provider = "groq", model = "openai/gpt-oss-20b")

  configs_list <- list(config1, config2)
  messages <- "Explain quantum computing"

  setup_llm_parallel(workers = 4, verbose = TRUE)
  results <- call_llm_compare(configs_list, messages)
  reset_llm_parallel(verbose = TRUE)
} # }