Agreement among coders or replicates

A thin, shape-aware wrapper around LLMR::llm_agreement(). Pass either a gold_set() whose coders columns hold individual human codings (the human-human reliability a methods section must report), or any data frame plus the label columns to compare (e.g. model replicates from code_corpus(), or one human column and one model column).

Usage

coder_agreement(x, cols = NULL)

Arguments

x

A gold_set() or a data frame.

cols

For data frames: character vector of label columns. Ignored for gold sets (their coders columns are used).

Value

An llmr_agreement object (mean pairwise agreement, Krippendorff's alpha, per-unit majorities); see LLMR::llm_agreement().