While transformers have pioneered attention-driven architectures as a
cornerstone of research, their dependence on explicitly contextual information
underscores limitations in their abilities to tacitly learn overarching textual
themes. This study investigates social media data as a source of distributed
patterns, challenging the heuristic paradigm of performance benchmarking. In
stark contrast to networks that rely on capturing complex long-term
dependencies, models of online data inherently lack structure and are forced to
learn underlying patterns in the aggregate. To properly represent these
abstract relationships, this research dissects empirical social media corpora
into their elemental components and analyzes over two billion tweets across
population-dense locations. Exploring the relationship between location and
vernacular in Twitter data, we employ Bag-of-Words models specific to each city
and evaluate their respective representation. This demonstrates that hidden
insights can be uncovered without the crutch of advanced algorithms and
demonstrates that even amidst noisy data, geographic location has a
considerable influence on online communication. This evidence presents tangible
insights regarding geospatial communication patterns and their implications in
social science. It also challenges the notion that intricate models are
prerequisites for pattern recognition in natural language, aligning with the
evolving landscape that questions the embrace of absolute interpretability over
abstract understanding. This study bridges the divide between sophisticated
frameworks and intangible relationships, paving the way for systems that blend
structured models with conjectural reasoning.

分析社交媒体数据中的位置和口语与 Twitter 数据之间的关系，采用每个城市特定的词袋模型并评估它们的表现。揭示了地理位置对在线交流的重大影响，证明了即使在嘈杂的数据中，也可以发现隐藏的见解。这项研究将复杂的模型与抽象理解结合起来，为将结构化模型与推测性推理相融合的系统开辟了道路。

超越注意范式：从地理空间社交媒体数据中的隐性学习

Transcending the Attention Paradigm: Implicit Learning from Geospatial  Social Media Data

Classical Chinese poetry is a jewel in the treasure house of Chinese culture.
Previous poem generation models only allow users to employ keywords to
interfere the meaning of generated poems, leaving the dominion of generation to
the model. In this paper, we propose a novel task of generating classical
Chinese poems from vernacular, which allows users to have more control over the
semantic of generated poems. We adapt the approach of unsupervised machine
translation (UMT) to our task. We use segmentation-based padding and
reinforcement learning to address under-translation and over-translation
respectively. According to experiments, our approach significantly improve the
perplexity and BLEU compared with typical UMT models. Furthermore, we explored
guidelines on how to write the input vernacular to generate better poems. Human
evaluation showed our approach can generate high-quality poems which are
comparable to amateur poems.

本文提出了一种利用非文言文生成古典诗歌的新任务，并采用了无监督机器翻译的方法，在语义上给予用户更多控制权，通过基于分词的填充和强化学习等方法，取得了很好的实验效果，同时探索了如何提高生成诗歌的输入口诀，得到了高质量的诗歌生成。