Large language models are increasingly used in humanities and communication
research, but their role remains ambivalent. They can support interpretive work, yet they
can also encourage the delegation of tasks that traditionally depend on human judgment.
A central concern is that, when LLMs are used to replace rather than support
interpretation, research becomes less grounded in shared deliberation and harder to
reproduce, evaluate, and contest. One reason for this is that interpretive work is often
offloaded to LLMs without sufficient control over the intermediate steps or adequate
insight into how decisions are produced. Against this background, it becomes important
to examine whether a more tightly controlled form of human–LLM collaboration can offer
a useful first step within a restricted application domain.
This presentation reports results from a study on the automatic detection of frames in
news headlines. It proposes a codebook-driven approach in which LLMs are treated not
as autonomous interpreters, but as annotator-colleagues within a structured, humancontrolled
workflow. Instead of asking a model to identify frames in a single opaque step,
the task is broken down into a series of binary annotation decisions derived from a
qualitative codebook. Models evaluate whether a headline matches specific framerelated
statements, and these judgments are then aggregated across multiple opensource
LLMs. Triangulation is used to reduce dependence on any single output and to
make disagreement visible. Using a corpus of news headlines, the study compares this
binary-question approach with full-description prompting and sentence-similarity
methods.
The argument is not that LLMs can solve framing analysis, but that carefully constrained
collaboration may offer a more adequate way of integrating them into interpretive
research. By preserving human control over the codebook, intermediate questions, and
aggregation procedure, the approach seeks to improve interpretability and reliability
while remaining critical of the limits, biases, and residual opacity that still shape LLMbased
research.
Speaker: Juan Sebastiàn Olier