Mobile apps have embraced user privacy by moving their data processing to the
user's smartphone. Advanced machine learning (ML) models, such as vision
models, can now locally analyze user images to extract insights that drive
several functionalities. Capitalizing on this new processing model of locally
analyzing user images, we analyze two popular social media apps, TikTok and
Instagram, to reveal (1) what insights vision models in both apps infer about
users from their image and video data and (2) whether these models exhibit
performance disparities with respect to demographics. As vision models provide
signals for sensitive technologies like age verification and facial
recognition, understanding potential biases in these models is crucial for
ensuring that users receive equitable and accurate services.
We develop a novel method for capturing and evaluating ML tasks in mobile
apps, overcoming challenges like code obfuscation, native code execution, and
scalability. Our method comprises ML task detection, ML pipeline
reconstruction, and ML performance assessment, specifically focusing on
demographic disparities. We apply our methodology to TikTok and Instagram,
revealing significant insights. For TikTok, we find issues in age and gender
prediction accuracy, particularly for minors and Black individuals. In
Instagram, our analysis uncovers demographic disparities in the extraction of
over 500 visual concepts from images, with evidence of spurious correlations
between demographic features and certain concepts.

移动应用程序通过将数据处理移至用户的智能手机来保护用户隐私。利用先进的机器学习（ML）模型，如视觉模型，现在可以在本地分析用户图像以提取推动多种功能的见解。通过分析两个流行的社交媒体应用程序 TikTok 和 Instagram，揭示了（1）这两个应用中的视觉模型从用户的图像和视频数据中推测出的见解以及（2）这些模型在与人口统计学相关性方面是否存在性能差异。了解这些模型中的潜在偏见对于确保用户获得公平且准确的服务至关重要。我们开发了一种捕获和评估移动应用程序中 ML 任务的新方法，克服了代码混淆、本地代码执行和可扩展性等挑战。我们的方法包括 ML 任务检测、ML 流水线重建和 ML 性能评估，特别关注人口统计学的差异。我们将该方法应用于 TikTok 和 Instagram，揭示了重要的见解。对于 TikTok，我们发现了关于年龄和性别预测准确性的问题，特别是对未成年人和黑人个体而言。在 Instagram 中，我们的分析揭示了超过 500 个图像概念中的人口统计学差异，存在某些概念与人口统计学特征之间的虚假相关性。

一张图片胜过 500 个标签：Instagram 和 TikTok 本地机器学习模型中的人口统计差异案例研究

A Picture is Worth 500 Labels: A Case Study of Demographic Disparities  in Local Machine Learning Models for Instagram and TikTok

I investigate bias in terms of ChatGPT's college major recommendations for
students with various profiles, looking at demographic disparities in factors
such as race, gender, and socioeconomic status, as well as educational
disparities such as score percentiles. By constructing prompts for the ChatGPT
API, allowing the model to recommend majors based on high school student
profiles, I evaluate bias using various metrics, including the Jaccard
Coefficient, Wasserstein Metric, and STEM Disparity Score. The results of this
study reveal a significant disparity in the set of recommended college majors,
irrespective of the bias metric applied.

通过构建聊天 GPT API 的提示，评估使用各种指标的偏差，包括 Jaccard 系数、Wasserstein 度量和 STEM 差异分数，揭示了建议的大学专业存在显著差异，无论应用何种偏差指标。