Pedestrian trajectory modelling in an urban complex is challenging because pedestrians can have many possible destinations, such as shops, escalators, and attractions. Moreover, weather and time-of-day may affect pedestrian behavior. In this paper, a new weather-time-trajectory fusion network (WTTFNet) is proposed to improve the performance of baseline deep neural network architecture. By incorporating weather and time-of-day information as an embedding structure, a novel WTTFNet based on gate multimodal unit is used to fuse the multimodal information and deep representation of trajectories. A joint loss function based on focal loss is used to co-optimize both the deep trajectory features and final classifier, which helps to improve the accuracy in predicting the intended destination of pedestrians and hence the trajectories under possible scenarios of class imbalances. Experimental results using the Osaka Asia and Pacific Trade Center (ATC) dataset shows improved performance of the proposed approach over state-of-the-art algorithms by 23.67% increase in classification accuracy, 9.16% and 7.07% reduction of average and final displacement error. The proposed approach may serve as an attractive approach for improving existing baseline trajectory prediction models when they are applied to scenarios with influences of weather-time conditions. It can be employed in numerous applications such as pedestrian facility engineering, public space development and technology-driven retail.

通过融合天气和时间等信息，借助于多模态单元的嵌入结构，提出了一种新的天气-时间-轨迹融合网络(WTTFNet)，用于改进基准深度神经网络架构，并采用联合损失函数和基于聚焦损失，通过优化深度轨迹特征和最终分类器，提高了对行人目的地和轨迹的准确性，该方法在与现有算法相比的实验结果中表现出23.67％的分类准确度提高，平均和最终位移误差分别降低了9.16％和7.07％，可应用于天气和时间等条件有影响的场景，用于行人设施工程、公共空间开发和技术驱动的零售等众多应用。

WTTFNet：城市复杂环境中行人轨迹预测的气象-时间-轨迹融合网络