Deep Learning (DL) is increasingly being integrated into Web applications
through a method known as "in-browser inference", where the DL processes occur
directly within Web browsers. However, the actual performance of this method
and its effect on user experience quality (QoE) is not well-understood. This
gap in knowledge necessitates new forms of QoE measurement, going beyond
traditional metrics such as page load time. To address this, we conducted the
first extensive performance evaluation of in-browser inference. We introduced
new metrics for this purpose: responsiveness, smoothness, and inference
accuracy.
Our thorough study included 9 widely-used DL models and tested them across 50
popular PC Web browsers. The findings show a significant latency issue with
in-browser inference: it's on average 16.9 times slower on CPU and 4.9 times
slower on GPU than native inference methods. Several factors contribute to this
latency, including underused hardware instruction sets, inherent delays in the
runtime environment, resource competition within the browser, and
inefficiencies in software libraries and GPU abstractions.
Moreover, in-browser inference demands a lot of memory, sometimes up to 334.6
times more than the size of the DL models themselves. This excessive memory
usage is partly due to suboptimal memory management. Additionally, we noticed
that in-browser inference increases the time it takes for graphical user
interface (GUI) components to load in web browsers by a significant 67.2\%,
which severely impacts the overall QoE for users of web applications that
depend on this technology.

深度学习在 Web 应用中的 “浏览器内推理” 方法正在被广泛应用，然而该方法的实际性能以及对用户体验质量（QoE）的影响尚未得到很好的了解。本研究通过对广泛使用的 9 个深度学习模型在 50 个流行的 PC Web 浏览器上的测试，提出了新的性能指标，包括响应速度、流畅度和推理准确性，研究结果显示浏览器内推理存在显著的延迟问题，相比本地推理方法，CPU 平均延迟慢 16.9 倍，GPU 平均延迟慢 4.9 倍，并且浏览器内推理对内存的需求非常大，有时达到 DL 模型本身大小的 334.6 倍，这导致了内存管理的亚最优性。此外，浏览器内推理还显著增加了 Web 浏览器中图形用户界面（GUI）组件加载的时间，对依赖该技术的 Web 应用用户的整体 QoE 造成严重影响。