In this episode, we explore Ex3, an innovative writing framework powered by large language models (LLMs) that aims to revolutionize long-form text generation. The episode delves into the challenges of using AI for narrative creation, particularly the shortcomings of traditional hierarchical generation methods in producing engaging, cohesive stories. Ex3 offers a fresh approach with its three-stage process: Extracting, Excelsior, and Expanding.
• Extracting begins by analyzing raw novel data, focusing on plot structure and character development. It groups text by semantic similarity, summarizes chapters hierarchically, and extracts key entity information to maintain coherence across the narrative.
• The Excelsior stage fine-tunes the LLM by creating an instruction-following dataset based on the extracted information, enhancing the model's ability to generate text aligned with a specific genre’s style and structure.
• Expanding introduces a depth-first writing mode, where the LLM generates novel text incrementally, building on the learned structure and entity information to craft a detailed and immersive story.
The episode wraps up with an evaluation of Ex3, comparing it to traditional methods using human assessments and automated metrics. It highlights Ex3's success in producing high-quality, long-form narratives while also discussing its current limitations, such as the need for better revision mechanisms and its focus on Chinese novels. Finally, the episode looks ahead to potential future developments in AI-driven storytelling.
https://arxiv.org/pdf/2408.08506