Infants' ability to recognize and categorize objects develops gradually. The
second year of life is marked by both the emergence of more semantic visual
representations and a better understanding of word meaning. This suggests that
language input may play an important role in shaping visual representations.
However, even in suitable contexts for word learning like dyadic play sessions,
caregivers utterances are sparse and ambiguous, often referring to objects that
are different from the one to which the child attends. Here, we systematically
investigate to what extent caregivers' utterances can nevertheless enhance
visual representations. For this we propose a computational model of visual
representation learning during dyadic play. We introduce a synthetic dataset of
ego-centric images perceived by a toddler-agent that moves and rotates toy
objects in different parts of its home environment while hearing caregivers'
utterances, modeled as captions. We propose to model toddlers' learning as
simultaneously aligning representations for 1) close-in-time images and 2)
co-occurring images and utterances. We show that utterances with statistics
matching those of real caregivers give rise to representations supporting
improved category recognition. Our analysis reveals that a small
decrease/increase in object-relevant naming frequencies can drastically impact
the learned representations. This affects the attention on object names within
an utterance, which is required for efficient visuo-linguistic alignment.
Overall, our results support the hypothesis that caregivers' naming utterances
can improve toddlers' visual representations.

研究表明，照顾者的命名话语可以改善婴儿的视觉表征能力。