AI tool forecasts development of ongoing written stories

Technology News | September 1, 2021

By Rich Pell

The tool uses a predictive algorithm that first looks at the preceding story and then predicts the “semantic frames” – each of which represents a cluster of concepts and related knowledge – that might occur in the next 10, 100, or even 1,000 sentences in an ongoing story. Their approach, say the researchers, could help authors who are experiencing writer’s block develop the next section of novel-length works.

Unlike current automated text generated methods, the new approach could help authors to craft language for the follow-up story arc beyond the scope of a few sentences – a limitation of existing models.

“These creative writing tasks seem nearly impossible to fully automate,” says Kenneth Huang, assistant professor of information sciences and technology. “The reason that we are tackling these very creative tasks is to push the boundaries of AI and natural language processing. Developing solutions for challenging creative tasks will teach us about the capacity and limitations of the current computational techniques, and so that we can further improve computer science.”

While existing models can generate a full story, they are only tested and proven to be successful on short works of 15 sentences or less. The researchers say they wanted to develop a tool that could help authors who write novels, which are typically 50,000 words or more.

“When providing longer text prediction,” says Chieh-Yang Huang, doctoral student of informatics, “we essentially provide follow-up ideas to help novelists to plan their story and set up goals instead of generating detailed stories for them. We envision that in the future we can provide various ideas to stimulate novelists to brainstorm different story arcs.”

The researchers’ framework – called semantic frame forecast – breaks a long narrative down into a sequence of text blocks, each containing a fixed number of sentences. The frequency of the occurrence of each semantic frame is then calculated.

Then, the text is converted to a vector – numerical data understood by a machine – where each dimension denotes the frequency of one frame. It is then computed to quantify the number of times a semantic frame appears and signifies its importance. Finally, the model inputs a fixed number of text blocks and predicts the semantic frame for the forthcoming block.

To make the output understandable to humans, the resulting vector is converted back from a set of numbers to a word cloud. Online crowd workers tested and confirmed the representativeness and specificity of the produced word clouds. Authors, say the researchers, could use the tool by feeding a part of their already-written text into the system to generate a set of word clouds with suggested nouns, verbs, and adjectives to inspire them when crafting the next part of their story.

The researchers say they tested their model on a dataset of nearly 5,000 fictional books and measured the tool’s effect of frame representation for different context lengths, varying the story block lengths between five and 1,000 sentences. In addition, they tested semantic frame forecast on nearly 8,000 scholarly articles using human-annotated abstracts from the CODA-19 dataset, highlighting the tool’s potential impact in nonfiction applications.

“It shows the generalizability of the technology,” says Huang. “Our approach works not only in stories, but also in scientific articles. If we can do it on both scientific papers and novels, we could probably do it on news and on other genres.”

The experiment, say the researchers, shows that forecasting forthcoming semantic frames is challenging but possible. The researchers say they plan to incorporate semantic frame forecast into a crowd-powered system that they previously developed, which enables writers to elicit story ideas from the online crowd, to further study how the tool can be used to support authors.

“If an automated system can augment human creativity, it will be impactful,” says Huang. “Even if the author doesn’t directly use what is generated, the machine’s outputs could inspire something that the writer didn’t think of before.”

The work was presented at the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), held virtually in early June.