Research

AI safety, governance & narrative

Two threads, one practice: evaluating and governing frontier AI, and reading culture computationally. Both treat the humanities as a source of method, not decoration.

AI safety & governance

As co-founder and technical lead of the team representing the Modern Language Association at NIST CAISI (the Center for AI Standards and Innovation), Jon works on LLM evaluation, red-teaming, and ethical auditing. The team's ethics-based audit results were presented during the opening keynote at the consortium's first plenary at the University of Maryland. With collaborators at Oxford and elsewhere he co-authored a comparative study of global AI regulation across the EU, China, and the US (doi:10.48550/arXiv.2410.21279), and was among the authors of the open-source generative-AI position paper accepted as an ICML 2024 oral presentation.

Computational narrative & digital humanities

One question runs through this thread: what would it take to measure the shape of a story reliably enough to argue about it? The answer has been a decade of building the instruments that make computational reading of narrative testable. “Middle reading” (2019) named a position between distant and close reading; SentimentArcs (2021) replaced the single trusted model with an ensemble and brought dynamic time warping to diachronic sentiment analysis of narrative, so arcs of unequal length could be compared by distance; eXplainable AI with GPT-4 (2023) made story trajectories explainable and extended that comparability to sentence-level sequences; MultiSentimentArcs (2024) carried the method across modalities, from text to film.

Jon and Katherine Elkins were also among the first to empirically evaluate GPT-3 for creative writing, and have written on what AI means for Narrative studies and on a human-centered AI curriculum. The work pairs real model evaluation with literary and ethical questions. Several of these threads are co-authored with Elkins, whose complementary program asks what stays the same and what changes when stories travel across languages, cultures, and time; see her research page.

Novelty boundary

What the work established, and when

Firsts are stated flatly; the dates carry the priority, and the Reception page documents the uptake that followed.

  • 2019: “Middle reading.” “Can Sentiment Analysis Reveal Structure in a Plotless Novel?” named a third position between distant reading (pure computation) and close reading (pure interpretation), and was the first to ask whether sentiment methods survive nonlinear narrative, the question computational literary studies still benchmarks against.
  • 2021: Ensembles, and arcs you can measure the distance between. SentimentArcs established that diachronic sentiment analysis of narrative needs an ensemble: dozens of models weighed against one another, with smoothing and model selection treated as interpretive choices rather than neutral preprocessing. It brought dynamic time warping into diachronic sentiment analysis, using DTW to compute the distance between two whole arcs while absorbing the temporal shifts and stretches that separate otherwise similar story shapes. Paired with LTTB downsampling, which normalizes every arc to the same number of points while keeping its peaks and valleys, this is the step that lets arcs of unequal length be compared and clustered by distance. The paper also reported that state-of-the-art transformers can struggle to find narrative arcs, a limit of scale, not of tuning.
  • 2023: Explainability and comparability. eXplainable AI with GPT-4 for Story Analysis and Generation brought XAI to narrative, where the technique had been largely confined to image classifiers, and supplied the sentence-level story-trajectory methodology together with the agglomeration and comparability solutions that let narratives of unequal length be compared on shared coordinates.
  • 2024: Across modalities. MultiSentimentArcs made cross-modal coherence measurable: whether the arc a film’s dialogue traces and the arc its images trace are in fact the same arc.

Together these make the trajectory of a story something you can measure and reproduce, including in later work comparing stories across languages and cultures.

Field-founding priority

Katherine Elkins and Jon Chun founded the world's first human-centered AI curriculum and lab at Kenyon College in 2016. Dates establish priority; the Reception page traces how the research that followed is cited across fields, and Human-Centered AI gives the definition and Stanford comparison.

Methods & instruments

SentimentArcs (arXiv:2110.09454) is the open-source ensemble the method runs on: dozens of sentiment models passed over the same narrative and weighed against one another, rather than a single model trusted on faith. Maintained since 2019 at jon-chun/sentiment_arcs, it is the technical foundation of The Shapes of Stories (Cambridge University Press, 2022).

Its time-series pipeline is where the ensemble becomes measurable. Arcs are standardized, smoothed with a 10% moving average, and reduced by LTTB downsampling to a common number of points that preserves peaks, valleys, and endpoints. Dynamic time warping then computes the distance between two whole arcs, absorbing the temporal shifts and stretches that separate otherwise similar story shapes, and those distances drive hierarchical clustering. The corpora range from roughly 1,400 to 13,000 data points per novel, so this is what makes arcs of unequal length comparable at all.

Narrative translation, cultural transmission, and computational philology → · The full technical lineage →

MultiSentimentArcs, in Frontiers in Computer Science, extends the method from text to film. The multimodal approach measures whether the sentiment arc recovered from a film’s dialogue and the arc recovered from its images are in fact the same arc, making cross-modal coherence a quantity that can be measured rather than asserted.

AI-LIT is the companion open repository for literary AI workflows. Katherine Elkins used it for text processing and visualization in In Search of a Translator (Frontiers in Computer Science 6:1444021, 13 August 2024; doi:10.3389/fcomp.2024.1444021), which develops an AI-assisted method for comparing a literary original with multiple translations across whole-narrative time, asking which dimensions of style and reading experience persist and which are altered or lost in translation.

Research threads

  • NIST CAISI evaluation. LLM evaluation and red-teaming for standards-facing work, with ethical auditing as the through-line.
  • Comparative AI regulation. Mapping policy approaches across the EU, China, and the US with Christian Schroeder de Witt and Katherine Elkins.
  • Computational narrative. From the GPT-3 Writer's Turing Test article to narrative theory, sentiment arcs, and AI-assisted interpretation.
  • Open-source generative AI. Risks and opportunities of open-source generative AI, including the ICML 2024 position paper.
  • Archival Intelligence. Applied humane-studies methods for cultural archives, multilingual newspapers, and early jazz artifacts.

Selected publications

Reception

The reception is strongest when described as specific scholarly uptake rather than as a generic citation count:

See Google Scholar →

FAQ

What does Jon Chun research?

AI safety, governance, and evaluation (LLM red-teaming and ethical auditing at NIST CAISI, and comparative global AI regulation), alongside computational narrative across the humanities.

What is his role at NIST CAISI?

He is co-founder and technical lead of the team representing the Modern Language Association at the NIST Center for AI Standards and Innovation, working on LLM evaluation and red-teaming.

Where has the research appeared?

Venues including ICML (a 2024 oral presentation), the Journal of Cultural Analytics, Narrative, and the International Journal of Humanities and Arts Computing.

Who founded Kenyon's human-centered AI curriculum and lab?

Katherine Elkins and Jon Chun founded the world's first human-centered AI curriculum and lab at Kenyon College in 2016. See how the work is cited across fields →

Building → Teaching → Reception →