MultiSentimentArcs
Extends narrative trajectory analysis from text to film, and makes cross-modal coherence measurable: whether the arc a film's dialogue traces and the arc its images trace are in fact the same arc.
- Full title. MultiSentimentArcs: a novel method to measure coherence in multimodal sentiment analysis for long-form narratives in film
- Author. Jon A. Chun
- Venue. Frontiers in Computer Science 6:1444549
- Published. 24 October 2024
- Identifier. doi:10.3389/fcomp.2024.1444549
- Access. Open access
Summary
A film tells its story twice at once: in what characters say, and in what the camera shows. Those two channels can agree, or they can pull against each other. MultiSentimentArcs extends the ensemble trajectory method from text to film and asks whether the sentiment arc recovered from a film's dialogue and the arc recovered from its images are in fact the same arc. It is a multimodal method in the sense of text and image, not multiple languages, and it uses a human-in-the-loop design in place of a single opaque model. Cross-modal coherence becomes a quantity that can be measured rather than asserted.
What this work contributed
- Trajectory analysis across modalities. The narrative-trajectory method is carried from text to film.
- A measure of cross-modal coherence. Agreement between a film's dialogue arc and its image arc is made measurable rather than asserted.
- Human-in-the-loop interpretation. Interpretive choices stay visible instead of being absorbed into a single opaque model.
Relationship to earlier and later work
It is the most recent of the narrative instruments, extending the ensemble pipeline of SentimentArcs (2021) and the trajectory methodology of eXplainable AI with GPT4 (2023). See Narrative trajectories & DTW for the lineage and Reception for uptake.
Cite
Plain text. Chun, Jon. “MultiSentimentArcs: a novel method to measure coherence in multimodal sentiment analysis for long-form narratives in film.” Frontiers in Computer Science 6 (2024): 1444549.
BibTeX.
@article{chun2024multisentimentarcs,
title = {MultiSentimentArcs: a novel method to measure coherence in multimodal
sentiment analysis for long-form narratives in film},
author = {Chun, Jon},
journal = {Frontiers in Computer Science},
volume = {6},
pages = {1444549},
year = {2024},
doi = {10.3389/fcomp.2024.1444549}
}