With the rapid development of social media such as Twitter and Weibo,
detecting keywords from a huge volume of text data streams in real-time has
become a critical problem. The keyword detection problem aims at searching
important information from massive text data to reflect the most important
events or topics. However, social media data usually has unique features: the
documents are usually short, the language is colloquial, and the data is likely
to have significant temporal patterns. Therefore, it could be challenging to
discover critical information from these text streams. In this paper, we
propose a novel method to address the keyword detection problem in social
media. Our model combines the Inverse Document Frequency (IDF) and Latent
Dirichlet Allocation (LDA) models to better cope with the distinct attributes
of social media data, such as the number of likes, comments, and retweets. By
weighting the importance of each document based on these attributes, our method
can effectively detect more representative keywords over time. Comprehensive
experiments conducted under various conditions on Weibo data illustrate that
our approach outperforms the baselines in various evaluation metrics, including
precision and recall for multiple problem settings.

本文提出了一种基于 IDF 和 LDA 模型相结合的方法，以更好地应对社交媒体数据的不同属性，通过基于点赞数、评论数和转发数等属性来加权每个文档的重要性，从而有效地检测出越来越具有代表性的关键词，实验证明该方法在各种评估指标，包括多个问题设置的精度和召回率等方面优于基准。

基于社交媒体属性的关键词检测：Sina 微博上的 IDF-LDA 模型应用

Utilizing Social Media Attributes for Enhanced Keyword Detection: An  IDF-LDA Model Applied to Sina Weibo

This paper introduces semantic frame forecast, a task that predicts the
semantic frames that will occur in the next 10, 100, or even 1,000 sentences in
a running story. Prior work focused on predicting the immediate future of a
story, such as one to a few sentences ahead. However, when novelists write long
stories, generating a few sentences is not enough to help them gain high-level
insight to develop the follow-up story. In this paper, we formulate a long
story as a sequence of "story blocks," where each block contains a fixed number
of sentences (e.g., 10, 100, or 200). This formulation allows us to predict the
follow-up story arc beyond the scope of a few sentences. We represent a story
block using the term frequencies (TF) of semantic frames in it, normalized by
each frame's inverse document frequency (IDF). We conduct semantic frame
forecast experiments on 4,794 books from the Bookcorpus and 7,962 scientific
abstracts from CODA-19, with block sizes ranging from 5 to 1,000 sentences. The
results show that automated models can forecast the follow-up story blocks
better than the random, prior, and replay baselines, indicating the task's
feasibility. We also learn that the models using the frame representation as
features outperform all the existing approaches when the block size is over 150
sentences. The human evaluation also shows that the proposed frame
representation, when visualized as word clouds, is comprehensible,
representative, and specific to humans. Our code is available at
this https URL

本文介绍语义框架预测技术，通过将长篇小说拆解为一系列固定长度的 “故事块”，使用语义框架频次标准化的方法，自动预测出后续故事情节。实验证明，当块长度超过 150 句时，此技术的自动化预测效果显著优于其他现有的方法。