Following the recent release of various Artificial Intelligence (AI) based
Conversation Agents (CAs), adolescents are increasingly using CAs for
interactive knowledge discovery on sensitive topics, including mental and
sexual health topics. Exploring such sensitive topics through online search has
been an essential part of adolescent development, and CAs can support their
knowledge discovery on such topics through human-like dialogues. Yet,
unintended risks have been documented with adolescents' interactions with
AI-based CAs, such as being exposed to inappropriate content, false
information, and/or being given advice that is detrimental to their mental and
physical well-being (e.g., to self-harm). In this position paper, we discuss
the current landscape and opportunities for CAs to support adolescents' mental
and sexual health knowledge discovery. We also discuss some of the challenges
related to ensuring the safety of adolescents when interacting with CAs
regarding sexual and mental health topics. We call for a discourse on how to
set guardrails for the safe evolution of AI-based CAs for adolescents.

通过对青少年与基于人工智能（AI）的会话代理之间的互动知识发现过程中的意外风险的讨论，本文呼吁对为保障青少年的安全与 AI 代理的进化进行讨论。

面向安全演化的基于人工智能（AI）的对话代理系统，支持青少年心理和性健康知识探索

Toward Safe Evolution of Artificial Intelligence (AI) based  Conversational Agents to Support Adolescent Mental and Sexual Health  Knowledge Discovery

Dialogue safety remains a pervasive challenge in open-domain human-machine
interaction. Existing approaches propose distinctive dialogue safety taxonomies
and datasets for detecting explicitly harmful responses. However, these
taxonomies may not be suitable for analyzing response safety in mental health
support. In real-world interactions, a model response deemed acceptable in
casual conversations might have a negligible positive impact on users seeking
mental health support. To address these limitations, this paper aims to develop
a theoretically and factually grounded taxonomy that prioritizes the positive
impact on help-seekers. Additionally, we create a benchmark corpus with
fine-grained labels for each dialogue session to facilitate further research.
We analyze the dataset using popular language models, including BERT-base,
RoBERTa-large, and ChatGPT, to detect and understand unsafe responses within
the context of mental health support. Our study reveals that ChatGPT struggles
to detect safety categories with detailed safety definitions in a zero- and
few-shot paradigm, whereas the fine-tuned model proves to be more suitable. The
developed dataset and findings serve as valuable benchmarks for advancing
research on dialogue safety in mental health support, with significant
implications for improving the design and deployment of conversation agents in
real-world applications. We release our code and data here:
this https URL

通过开发具有理论和事实基础的聚焦于帮助寻求者积极影响的分类法以及创建具有细粒度标签的基准语料库，本研究在心理健康支持对话中分析使用 BERT-base、RoBERTa-large 和 ChatGPT 等流行语言模型以检测和理解不安全回应，并揭示 ChatGPT 在零样本和少样本范式中无法检测具有详细定义的安全类别，而经过微调的模型更加适用，为心理健康支持对话的对话安全研究提供了有价值的基准，并对改善真实应用中对话代理的设计和部署产生了重大影响。

精神健康支持中对话安全的基准测试

A Benchmark for Understanding Dialogue Safety in Mental Health Support

Conversation agents fueled by Large Language Models (LLMs) are providing a
new way to interact with visual data. While there have been initial attempts
for image-based conversation models, this work addresses the underexplored
field of video-based conversation by introducing Video-ChatGPT. It is a
multimodal model that merges a video-adapted visual encoder with a LLM. The
model is capable of understanding and generating human-like conversations about
videos. We introduce a new dataset of 100,000 video-instruction pairs used to
train Video-ChatGPT acquired via manual and semi-automated pipeline that is
easily scalable and robust to label noise. We also develop a quantiative
evaluation framework for video-based dialogue models to objectively analyse the
strengths and weaknesses of proposed models. Our code, models, instruction-sets
and demo are released at this https URL

介绍了基于视觉编码器与大型语言模型相结合的 Video-ChatGPT 模型，用于理解和生成关于视频的人类对话，并介绍了使用手动和半自动管道获得的新数据集，可用于训练和评估基于视频的对话模型，并在定量评估框架下分析了该模型的优劣。