Coding, which targets compressing and reconstructing data, and intelligence,
often regarded at an abstract computational level as being centered around
model learning and prediction, interweave recently to give birth to a series of
significant progress. The recent trends demonstrate the potential homogeneity
of these two fields, especially when deep-learning models aid these two
categories for better probability modeling. For better understanding and
describing from a unified perspective, inspired by the basic generally
recognized principles in cognitive psychology, we formulate a novel problem of
Coding for Intelligence from the category theory view. Based on the three
axioms: existence of ideal coding, existence of practical coding, and
compactness promoting generalization, we derive a general framework to
understand existing methodologies, namely that, coding captures the intrinsic
relationships of objects as much as possible, while ignoring information
irrelevant to downstream tasks. This framework helps identify the challenges
and essential elements in solving the specific derived Minimal Description
Length (MDL) optimization problem from a broader range, providing opportunities
to build a more intelligent system for handling multiple tasks/applications
with coding ideas/tools. Centering on those elements, we systematically review
recent processes of towards optimizing the MDL problem in more comprehensive
ways from data, model, and task perspectives, and reveal their impacts on the
potential CfI technical routes. After that, we also present new technique paths
to fulfill CfI and provide potential solutions with preliminary experimental
evidence. Last, further directions and remaining issues are discussed as well.
The discussion shows our theory can reveal many phenomena and insights about
large foundation models, which mutually corroborate with recent practices in
feature learning.

编码和智能密切相关，基于类别论的视角提出了 “编码为智能” 问题，并推导了一个通用框架来理解现有方法。该框架帮助识别解决多任务／应用的最小描述长度（MDL）优化问题的挑战和关键元素，提供了构建更智能系统的机会。通过系统回顾最近在数据、模型和任务层面上优化 MDL 问题的方法以及初步实验结果，提供了潜在的解决路径和技术方向。讨论还探讨了进一步的方向和存在的问题，并与特征学习中的实践相互印证，揭示了大基础模型的现象和见解。

从分类的角度编码智能

Coding for Intelligence from the Perspective of Category

Machine Learning (ML) and Algorithmic Information Theory (AIT) look at
Complexity from different points of view. We explore the interface between AIT
and Kernel Methods (that are prevalent in ML) by adopting an AIT perspective on
the problem of learning kernels from data, in kernel ridge regression, through
the method of Sparse Kernel Flows. In particular, by looking at the differences
and commonalities between Minimal Description Length (MDL) and Regularization
in Machine Learning (RML), we prove that the method of Sparse Kernel Flows is
the natural approach to adopt to learn kernels from data. This paper shows that
it is not necessary to use the statistical route to derive Sparse Kernel Flows
and that one can directly work with code-lengths and complexities that are
concepts that show up in AIT.

通过采用 AIT（算法信息论）在从数据中学习核函数的问题上的角度，我们探索了机器学习（ML）和算法信息论（AIT）之间的接口，特别是通过考察最小描述长度（MDL）和机器学习中的正则化（RML）之间的差异和共同点，证明了稀疏核流方法是自然的方法来学习核函数。本文表明，从推导稀疏核流的统计途径并非必要，而是可以直接使用出现在 AIT 中的代码长度和复杂性相关的概念。

算法信息论与机器学习的桥梁：一种新的核学习方法

Bridging Algorithmic Information Theory and Machine Learning: A New  Approach to Kernel Learning

We explore the issue of refining an existent Bayesian network structure using
new data which might mention only a subset of the variables. Most previous
works have only considered the refinement of the network's conditional
probability parameters, and have not addressed the issue of refining the
network's structure. We develop a new approach for refining the network's
structure. Our approach is based on the Minimal Description Length (MDL)
principle, and it employs an adapted version of a Bayesian network learning
algorithm developed in our previous work. One of the adaptations required is to
modify the previous algorithm to account for the structure of the existent
network. The learning algorithm generates a partial network structure which can
then be used to improve the existent network. We also present experimental
evidence demonstrating the effectiveness of our approach.

使用最小描述长度 (MDL) 原则和贝叶斯网络学习算法的改进版本，我们探讨了对已存在的贝叶斯网络的结构进行精炼的问题，同时我们通过实验证据表明了我们的方法的有效性。