State-of-the-art language models are autoregressive and operate on subword
units known as tokens. Specifically, one must encode the conditioning string
into a list of tokens before passing to the language models for next-token
prediction. We show that, for encoding schemes such as maximum prefix matching,
tokenization induces a sampling bias that cannot be mitigated with more
training or data. To counter this universal problem, we propose a novel
algorithm to obtain unbiased estimates from a model that was trained on
tokenized data. Our method does not require finetuning the model, and its
complexity, defined as the number of model runs, scales linearly with the
sequence length. As a consequence, we show that one can simulate token-free
behavior from a tokenized language model. We empirically verify the correctness
of our method through a Markov-chain setup, where it accurately recovers the
transition probabilities, as opposed to the conventional method of directly
prompting tokens into the language model.

通过提出一种新颖的算法，我们可以从单词化数据中得到无偏估计，而不需要调整模型。通过 Markov 链设置，我们从标记化语言模型中精准恢复了转换概率。

理解和减轻语言模型中的分词偏差

Understanding and Mitigating Tokenization Bias in Language Models

This work introduces a novel R package for concise, informative summaries of
machine learning models.
We take inspiration from the summary function for (generalized) linear models
in R, but extend it in several directions:
First, our summary function is model-agnostic and provides a unified summary
output also for non-parametric machine learning models;
Second, the summary output is more extensive and customizable -- it comprises
information on the dataset, model performance, model complexity, model's
estimated feature importances, feature effects, and fairness metrics;
Third, models are evaluated based on resampling strategies for unbiased
estimates of model performances, feature importances, etc.
Overall, the clear, structured output should help to enhance and expedite the
model selection process, making it a helpful tool for practitioners and
researchers alike.

本研究介绍了一种新的 R 软件包，用于对机器学习模型进行简洁、信息丰富的总结，该软件包的总结函数灵感来自于 R 中（广义）线性模型的总结函数，但在几个方面进行了扩展：首先，我们的总结函数与模型无关，并提供一个统一的总结输出，适用于非参数机器学习模型；第二，总结输出更加全面和可定制，包括有关数据集、模型性能、模型复杂性、估计的特征重要性、特征效应和公平度量的信息；第三，使用重采样策略对模型性能、特征重要性等进行无偏估计来评估模型。总体而言，这种清晰、结构化的输出应该有助于提高和加快模型选择过程，成为从业者和研究人员的有用工具。

mlr3summary：机器学习模型的简洁和可解释摘要

mlr3summary: Concise and interpretable summaries for machine learning  models

Noisy data, non-convex objectives, model misspecification, and numerical
instability can all cause undesired behaviors in machine learning systems. As a
result, detecting actual implementation errors can be extremely difficult. We
demonstrate a methodology in which developers use an interactive proof
assistant to both implement their system and to state a formal theorem defining
what it means for their system to be correct. The process of proving this
theorem interactively in the proof assistant exposes all implementation errors
since any error in the program would cause the proof to fail. As a case study,
we implement a new system, Certigrad, for optimizing over stochastic
computation graphs, and we generate a formal (i.e. machine-checkable) proof
that the gradients sampled by the system are unbiased estimates of the true
mathematical gradients. We train a variational autoencoder using Certigrad and
find the performance comparable to training the same model in TensorFlow.

通过可交互的证明助手，开发者能够证明机器学习系统的正确性，这种方法暴露了所有的实现错误，并通过 Certigrad 实现了优化的随机计算图，并生成了一个机器可验证的证明，证明了系统采样的梯度是数学梯度的无偏估计方法。