Linear sequences of words are implicitly represented in our brains by
hierarchical structures that organize the composition of words in sentences.
Linguists formalize different frameworks to model this hierarchy; two of the
most common syntactic frameworks are Constituency and Dependency. Constituency
represents sentences as nested groups of phrases, while dependency represents a
sentence by assigning relations between its words. Recently, the pursuit of
intelligent machines has produced Language Models (LMs) capable of solving many
language tasks with a human-level performance. Many studies now question
whether LMs implicitly represent syntactic hierarchies. This thesis focuses on
producing constituency and dependency structures from LMs in an unsupervised
setting. I review the critical methods in this field and highlight a line of
work that utilizes a numerical representation for binary constituency trees
(Syntactic Distance). I present a detailed study on StructFormer (SF) (Shen et
al., 2021), which retrofits a transformer encoder architecture with a parser
network to produce constituency and dependency structures. I present six
experiments to analyze and address this field's challenges; experiments include
investigating the effect of repositioning the parser network within the SF
architecture, evaluating subword-based induced trees, and benchmarking the
models developed in the thesis experiments on linguistic tasks. Models
benchmarking is performed by participating in the BabyLM challenge, published
at CoNLL 2023 (Momen et al., 2023). The results of this thesis encourage
further development in the direction of retrofitting transformer-based models
to induce syntactic structures, supported by the acceptable performance of SF
in different experimental settings and the observed limitations that require
innovative solutions to advance the state of syntactic structure induction.

该论文研究了从语言模型中以无监督方式生成组成和依赖结构的问题，并通过一系列实验表明了其在语法结构识别方面的可行性和存在的限制。