For those experiencing severe-to-profound sensorineural hearing loss, the
cochlear implant (CI) is the preferred treatment. Augmented reality (AR) aided
surgery can potentially improve CI procedures and hearing outcomes. Typically,
AR solutions for image-guided surgery rely on optical tracking systems to
register pre-operative planning information to the display so that hidden
anatomy or other important information can be overlayed and co-registered with
the view of the surgical scene. In this paper, our goal is to develop a method
that permits direct 2D-to-3D registration of the microscope video to the
pre-operative Computed Tomography (CT) scan without the need for external
tracking equipment. Our proposed solution involves using surface mapping of a
portion of the incus in surgical recordings and determining the pose of this
structure relative to the surgical microscope by performing pose estimation via
the perspective-n-point (PnP) algorithm. This registration can then be applied
to pre-operative segmentations of other anatomy-of-interest, as well as the
planned electrode insertion trajectory to co-register this information for the
AR display. Our results demonstrate the accuracy with an average rotation error
of less than 25 degrees and a translation error of less than 2 mm, 3 mm, and
0.55% for the x, y, and z axes, respectively. Our proposed method has the
potential to be applicable and generalized to other surgical procedures while
only needing a monocular microscope during intra-operation.

通过进行 2D 到 3D 的注册和姿态估计，我们提出了一种不需要外部追踪设备，而可以将显微镜视频直接与术前计算机断层扫描（CT 扫描）进行配准的方法，从而为增强现实手术提供了准确且精确度较高的效果。

单目显微镜到 CT 图像配准：基于镫骨位姿估计的增强现实耳蜗植入手术

Monocular Microscope to CT Registration using Pose Estimation of the  Incus for Augmented Reality Cochlear Implant Surgery

ElectrodeNet, a deep learning based sound coding strategy for the cochlear
implant (CI), is proposed to emulate the advanced combination encoder (ACE)
strategy by replacing the conventional envelope detection using various
artificial neural networks. The extended ElectrodeNet-CS strategy further
incorporates the channel selection (CS). Network models of deep neural network
(DNN), convolutional neural network (CNN), and long short-term memory (LSTM)
were trained using the Fast Fourier Transformed bins and channel envelopes
obtained from the processing of clean speech by the ACE strategy. Objective
speech understanding using short-time objective intelligibility (STOI) and
normalized covariance metric (NCM) was estimated for ElectrodeNet using CI
simulations. Sentence recognition tests for vocoded Mandarin speech were
conducted with normal-hearing listeners. DNN, CNN, and LSTM based ElectrodeNets
exhibited strong correlations to ACE in objective and subjective scores using
mean squared error (MSE), linear correlation coefficient (LCC) and Spearman's
rank correlation coefficient (SRCC). The ElectrodeNet-CS strategy was capable
of producing N-of-M compatible electrode patterns using a modified DNN network
to embed maxima selection, and to perform in similar or even slightly higher
average in STOI and sentence recognition compared to ACE. The methods and
findings demonstrated the feasibility and potential of using deep learning in
CI coding strategy.

提出了一种基于深度学习的听神经植入体 (CI) 声音编码策略 ElectrodeNet，通过使用各种人工神经网络替代传统的包络检测来模拟先进的组合编码器（ACE）策略。在电子模拟中，使用 CI 模拟评估了 ElectrodeNet 的客观语音理解能力和主观分数。与 ACE 相比，基于深度神经网络（DNN）、卷积神经网络（CNN）和长短时记忆（LSTM）的 ElectrodeNets 在客观和主观分数上表现出强相关性。ElectrodeNet-CS 策略通过使用修改后的 DNN 网络嵌入最大选择来产生 N-of-M 兼容的电极模式，并且在 STOI 和语句识别方面表现出与 ACE 相似或略高的平均水平，证明了在 CI 编码策略中使用深度学习的可行性和潜力。

ElectrodeNet -- 一种基于深度学习的人工耳蜗声音编码策略

ElectrodeNet -- A Deep Learning Based Sound Coding Strategy for Cochlear  Implants

Prosody plays a vital role in verbal communication. Acoustic cues of prosody
have been examined extensively. However, prosodic characteristics are not only
perceived auditorily, but also visually based on head and facial movements. The
purpose of this report is to present a method for examining audiovisual prosody
using virtual reality. We show that animations based on a virtual human provide
motion cues similar to those obtained from video recordings of a real talker.
The use of virtual reality opens up new avenues for examining multimodal
effects of verbal communication. We discuss the method in the framework of
examining prosody perception in cochlear implant listeners.

本文介绍了一种使用虚拟现实检测音频视觉韵律的方法，并探讨了其在考察人工耳蜗听众韵律感知方面的应用。