Foundational Language Models (FLMs) have advanced natural language processing
(NLP) research. Current researchers are developing larger FLMs (e.g., XLNet,
T5) to enable contextualized language representation, classification, and
generation. While developing larger FLMs has been of significant advantage, it
is also a liability concerning hallucination and predictive uncertainty.
Fundamentally, larger FLMs are built on the same foundations as smaller FLMs
(e.g., BERT); hence, one must recognize the potential of smaller FLMs which can
be realized through an ensemble. In the current research, we perform a reality
check on FLMs and their ensemble on benchmark and real-world datasets. We
hypothesize that the ensembling of FLMs can influence the individualistic
attention of FLMs and unravel the strength of coordination and cooperation of
different FLMs. We utilize BERT and define three other ensemble techniques:
{Shallow, Semi, and Deep}, wherein the Deep-Ensemble introduces a
knowledge-guided reinforcement learning approach. We discovered that the
suggested Deep-Ensemble BERT outperforms its large variation i.e. BERTlarge, by
a factor of many times using datasets that show the usefulness of NLP in
sensitive fields, such as mental health.

基于大型 FLMs 的小型 FLMs 的潜力是基于同一基础的，通过集成技术可以影响 FLMs 的关注点，揭示不同 FLMs 的协作与合作的能力，我们发现深度集成的 BERT 通过使用心理健康等敏感领域的数据，相对于其大型版本 BERTlarge，具有倍数倍的性能优势。