Skip to contents

Generate Embeddings in Batches

Source: R/LLMR.R
get_batched_embeddings.Rd

A wrapper function that processes a list of texts in batches to generate embeddings, avoiding rate limits. This function calls call_llm_robust for each batch and stitches the results together and parses them (using parse_embeddings) to return a numeric matrix.

Usage

get_batched_embeddings(
  texts,
  embed_config,
  batch_size = 50,
  verbose = FALSE,
  tries = 5,
  wait_seconds = 2,
  backoff_factor = 3
)

Arguments

texts

Character vector of texts to embed. If named, the names will be used as row names in the output matrix.

embed_config

An llm_config object configured for embeddings.

batch_size

Integer. Number of texts to process in each batch. Default is 50. (Gemini's developer API embeds at most 100 texts per request; larger batches are split automatically.)

verbose

Logical. If TRUE, prints progress messages. Default is FALSE.

tries, wait_seconds, backoff_factor

Retry controls forwarded to call_llm_robust for each batch.

Value

A numeric matrix where each row is an embedding vector for the corresponding text. Columns are named v1, v2, ..., vK where K is the embedding dimension. If embedding fails for certain texts, those rows will be filled with NA values. The matrix will always have the same number of rows as the input texts. Returns NULL if no embeddings were successfully generated.

Batching, chunking, and row packing

"Batch" appears in three distinct senses in LLMR, and they are easy to keep apart once you note which path each belongs to. (1) The asynchronous provider Batch API (llm_batch_submit() and friends) defers a whole job for later delivery at a reduced price; it is the only deferred path here. (2) In the embedding path, get_batched_embeddings()'s batch_size sets how many texts go in one synchronous embedding request, bounded by the provider's per-call limit. (3) In the generative path, the .rows_per_prompt argument here packs several data rows into one generative prompt and parses them back into rows. Senses (2) and (3) are synchronous: batch_size and .rows_per_prompt control how work is grouped into requests, not the per-token rate, so the synchronous helpers do not themselves apply a provider discount. Pricing is a separate axis: several providers bill batched embeddings at a reduced rate through a dedicated async/batch tier (the embedding analogue of (1)), so embeddings are not categorically full-price; that discount is a property of the endpoint, not of batch_size. Row packing (3) can still lower total tokens by amortizing shared prompt overhead across rows, again without changing the rate.

See also

llm_config to create the embedding configuration. parse_embeddings to convert the raw response to a numeric matrix. Here batch_size is embedding chunking, the synchronous sense; it is not the asynchronous provider Batch API (llm_batch_submit) nor the generative row packing of llm_mutate (.rows_per_prompt).

Examples

if (FALSE) { # \dontrun{
  # Basic usage
  texts <- c("Hello world", "How are you?", "Machine learning is great")
  names(texts) <- c("greeting", "question", "statement")

  # The key is read from the VOYAGE_API_KEY environment variable.
  embed_cfg <- llm_config(
    provider = "voyage",
    model = "voyage-3.5-lite",
    embedding = TRUE
  )

  embeddings <- get_batched_embeddings(
    texts = texts,
    embed_config = embed_cfg,
    batch_size = 2
  )
} # }