With the increased attention to memristive-based in-memory analog computing
(IMAC) architectures as an alternative for energy-hungry computer systems for
machine learning applications, a tool that enables exploring their device- and
circuit-level design space can significantly boost the research and development
in this area. Thus, in this paper, we develop IMAC-Sim, a circuit-level
simulator for the design space exploration of IMAC architectures. IMAC-Sim is a
Python-based simulation framework, which creates the SPICE netlist of the IMAC
circuit based on various device- and circuit-level hyperparameters selected by
the user, and automatically evaluates the accuracy, power consumption, and
latency of the developed circuit using a user-specified dataset. Moreover,
IMAC-Sim simulates the interconnect parasitic resistance and capacitance in the
IMAC architectures and is also equipped with horizontal and vertical
partitioning techniques to surmount these reliability challenges. IMAC-Sim is a
flexible tool that supports a broad range of device- and circuit-level
hyperparameters. In this paper, we perform controlled experiments to exhibit
some of the important capabilities of the IMAC-Sim, while the entirety of its
features is available for researchers via an open-source tool.

本文介绍一种基于 Python 的 IMAC-Sim 电路级模拟器，用于设计空间中的电路仿真及模拟，该模拟器能够评估电路的准确性、功耗和延迟，并支持各种设备和电路级超参数。

IMAC-Sim: 基于电路的内存模拟仿真器，用于模拟模拟计算架构

IMAC-Sim: A Circuit-level Simulator For In-Memory Analog Computing  Architectures

We study the problem of efficient generative inference for Transformer
models, in one of its most challenging settings: large deep models, with tight
latency targets and long sequence lengths. Better understanding of the
engineering tradeoffs for inference for large Transformer-based models is
important as use cases of these models are growing rapidly throughout
application areas. We develop a simple analytical model for inference
efficiency to select the best multi-dimensional partitioning techniques
optimized for TPU v4 slices based on the application requirements. We combine
these with a suite of low-level optimizations to achieve a new Pareto frontier
on the latency and model FLOPS utilization (MFU) tradeoffs on 500B+ parameter
models that outperforms the FasterTransformer suite of benchmarks. We further
show that with appropriate partitioning, the lower memory requirements of
multiquery attention (i.e. multiple query heads share single key/value head)
enables scaling up to 32x larger context lengths. Finally, we achieve a
low-batch-size latency of 29ms per token during generation (using int8 weight
quantization) and a 76% MFU during large-batch-size processing of input tokens,
while supporting a long 2048-token context length on the PaLM 540B parameter
model.

该研究旨在提高 Transformers 模型的生成推理效率，并通过多维分区技术、低级优化等策略获得较佳的推理效率和 FLOPS 利用率权衡，从而支持 Token 的大批量处理和长文本生成。

高效扩展 Transformer 推理

Efficiently Scaling Transformer Inference

Tree search algorithms, such as branch-and-bound, are the most widely used
tools for solving combinatorial and nonconvex problems. For example, they are
the foremost method for solving (mixed) integer programs and constraint
satisfaction problems. Tree search algorithms recursively partition the search
space to find an optimal solution. In order to keep the tree size small, it is
crucial to carefully decide, when expanding a tree node, which question
(typically variable) to branch on at that node in order to partition the
remaining space. Numerous partitioning techniques (e.g., variable selection)
have been proposed, but there is no theory describing which technique is
optimal. We show how to use machine learning to determine an optimal weighting
of any set of partitioning procedures for the instance distribution at hand
using samples from the distribution. We provide the first sample complexity
guarantees for tree search algorithm configuration. These guarantees bound the
number of samples sufficient to ensure that the empirical performance of an
algorithm over the samples nearly matches its expected performance on the
unknown instance distribution. This thorough theoretical investigation
naturally gives rise to our learning algorithm. Via experiments, we show that
learning an optimal weighting of partitioning procedures can dramatically
reduce tree size, and we prove that this reduction can even be exponential.
Through theory and experiments, we show that learning to branch is both
practical and hugely beneficial.

使用机器学习确定任何一组划分程序的最佳加权，并通过实验证明，学习如何分叉在实践中非常有益，可以显著减小树的规模。