Skip to contents

Install a local model and the llama.cpp server

Source: R/local_install.R
llm_local_install.Rd

Downloads what a local model needs and records it, so that llm_config("llamacpp", model) works offline from then on: the llama.cpp server program for this computer's platform (from the official llama.cpp releases on GitHub) and the model's GGUF file (from Hugging Face). Each file is pinned to an exact version and checked against its sha256 after download; a file that fails the check is deleted. Before downloading, the function lists every file with its size, source, and license, and asks for agreement; in a non-interactive session accept_license = TRUE gives that agreement. Files already present are not downloaded again, so a second call makes no network request. Nothing is downloaded at any other time.

Usage

llm_local_install(
  model = "qwen3-0.6b",
  accept_license = FALSE,
  device = c("auto", "cpu", "vulkan"),
  dir = llm_local_dir(),
  verify = FALSE,
  force = FALSE,
  quiet = !interactive(),
  timeout = 3600
)

Arguments

model

A model name from llm_local_models(), or a specification from llm_local_model() for another GGUF file on Hugging Face. NULL installs only the server.

accept_license

TRUE to agree to the licenses of the downloaded files without being asked, as a non-interactive session requires.

device

"auto" (Metal on Apple silicon, otherwise CPU), "cpu", or "vulkan" (Linux and Windows).

dir

Directory for the files; see llm_local_dir().

verify

If TRUE, recompute the sha256 of files already installed.

force

If TRUE, download again even when the files are present.

quiet

If TRUE, show no download progress.

timeout

Seconds allowed for each download.

Value

The model name, invisibly.

Details

The server runs on this computer, listens only on 127.0.0.1, and requires a key generated for each session. It starts on the first call that needs it and stops when R exits (see llm_local_start()). On macOS with Apple silicon it uses the GPU through Metal; elsewhere it uses the CPU, or a GPU through Vulkan with device = "vulkan" on Linux and Windows.

See also

llm_local_models(), llm_local_status(), llm_local_uninstall(), llm_next_token(), llm_continue(), llm_score()

Examples

if (FALSE) { # \dontrun{
llm_local_install("qwen3-0.6b")
cfg <- llm_config("llamacpp", "qwen3-0.6b", temperature = 0)
call_llm(cfg, "What is the capital of France? One word.")

llm_local_install("bge-m3")
X <- get_batched_embeddings(c("budget", "\u0628\u0648\u062f\u062c\u0647"),
                            llm_config("llamacpp", "bge-m3"))
} # }