There is a growing interest in subject-specific predictions using deep neural
networks (DNNs) because real-world data often exhibit correlations, which has
been typically overlooked in traditional DNN frameworks. In this paper, we
propose a novel hierarchical likelihood learning framework for introducing
gamma random effects into the Poisson DNN, so as to improve the prediction
performance by capturing both nonlinear effects of input variables and
subject-specific cluster effects. The proposed method simultaneously yields
maximum likelihood estimators for fixed parameters and best unbiased predictors
for random effects by optimizing a single objective function. This approach
enables a fast end-to-end algorithm for handling clustered count data, which
often involve high-cardinality categorical features. Furthermore,
state-of-the-art network architectures can be easily implemented into the
proposed h-likelihood framework. As an example, we introduce multi-head
attention layer and a sparsemax function, which allows feature selection in
high-dimensional settings. To enhance practical performance and learning
efficiency, we present an adjustment procedure for prediction of random
parameters and a method-of-moments estimator for pretraining of variance
component. Various experiential studies and real data analyses confirm the
advantages of our proposed methods.

提出一种新的层级似然学习框架，将伽马随机效应引入 Poisson 深度神经网络，以捕捉输入变量的非线性效应和主题特定的聚类效应，从而提高预测性能。该方法通过优化单一目标函数同时获得固定参数的最大似然估计值和随机效应的最佳无偏预测器，为处理包含高基数分类特征的聚类计数数据提供了一个快速端到端算法。并且，可以很容易地将最新的网络架构实施到提出的 h - 似然框架中，例如多头注意力层和稀疏最大函数，用于在高维环境中进行特征选择。为了提高实际性能和学习效率，还通过调整过程来预测随机参数，并使用矩量法估计方差分量进行预训练。各种实验研究和真实数据分析证实了我们提出方法的优势。