Given the high economic and environmental costs of using large vision or
language models, analog in-memory accelerators present a promising solution for
energy-efficient AI. While inference on analog accelerators has been studied
recently, the training perspective is underexplored. Recent studies have shown
that the "workhorse" of digital AI training - stochastic gradient descent (SGD)
algorithm converges inexactly when applied to model training on non-ideal
devices. This paper puts forth a theoretical foundation for gradient-based
training on analog devices. We begin by characterizing the non-convergent issue
of SGD, which is caused by the asymmetric updates on the analog devices. We
then provide a lower bound of the asymptotic error to show that there is a
fundamental performance limit of SGD-based analog training rather than an
artifact of our analysis. To address this issue, we study a heuristic analog
algorithm called Tiki-Taka that has recently exhibited superior empirical
performance compared to SGD and rigorously show its ability to exactly converge
to a critical point and hence eliminates the asymptotic error. The simulations
verify the correctness of the analyses.

基于模拟设备的梯度训练是一种能耗更低的人工智能解决方案，该论文通过研究模拟加速器上的梯度训练，提出了一个理论基础，并介绍了一种名为 Tiki-Taka 的启发式模拟算法，该算法能够快速收敛到关键点，消除了渐近误差。

基于模拟内存计算的精确基于梯度的训练

Towards Exact Gradient-based Training on Analog In-memory Computing

Artificial Intelligence (AI) has achieved significant advancements in
technology and research with the development over several decades, and is
widely used in many areas including computing vision, natural language
processing, time-series analysis, speech synthesis, etc. During the age of deep
learning, especially with the arise of Large Language Models, a large majority
of researchers' attention is paid on pursuing new state-of-the-art (SOTA)
results, resulting in ever increasing of model size and computational
complexity. The needs for high computing power brings higher carbon emission
and undermines research fairness by preventing small or medium-sized research
institutions and companies with limited funding in participating in research.
To tackle the challenges of computing resources and environmental impact of AI,
Green Computing has become a hot research topic. In this survey, we give a
systematic overview of the technologies used in Green Computing. We propose the
framework of Green Computing and devide it into four key components: (1)
Measures of Greenness, (2) Energy-Efficient AI, (3) Energy-Efficient Computing
Systems and (4) AI Use Cases for Sustainability. For each components, we
discuss the research progress made and the commonly used techniques to optimize
the AI efficiency. We conclude that this new research direction has the
potential to address the conflicts between resource constraints and AI
development. We encourage more researchers to put attention on this direction
and make AI more environmental friendly.

人工智能在技术和研究方面取得了显著的进展，并广泛应用于计算视觉、自然语言处理、时间序列分析、语音合成等多个领域。然而，随着大型语言模型的出现，研究人员的注意力主要集中在追求最新的技术成果，导致模型规模和计算复杂性不断增加，需要更高的计算能力，进而产生更高的碳排放量，阻碍了资金有限的中小型研究机构和公司参与研究，并引发研究公平性的问题。为应对计算资源和人工智能的环境影响，绿色计算已成为一个热门的研究课题。在本调研中，我们对绿色计算中使用的技术进行了系统概述，并提出绿色计算的框架，将其分为四个关键组成部分：(1) 绿色度量指标，(2) 能效人工智能，(3) 能效计算系统和 (4) 可持续发展的人工智能应用案例。针对每个组成部分，我们讨论了研究进展及常用的优化技术，结论表明这个新的研究方向有潜力解决资源限制与人工智能发展之间的冲突。我们鼓励更多的研究人员关注这个方向，使人工智能更加环保友好。