The artifacts silicon respondents are known for, measured from the responses themselves:
Arguments
- responses
A
panel_administer()result.
Value
A tibble: item_id, n, parse_failures, order_effect_p
(NA when order was not randomized or cells are too sparse).
Details
Option-order effects: for items administered with randomized option order, a chi-squared test of response against the order seen. With LLMs this is routinely significant; a result that survives LLMRcontent-style scrutiny should not depend on it.
Non-response: parse failures and refusals per item.