Abstract: In order to further explore the human mobility behavior pattern, this paper proposes a network model based on Siamese hierarchical attention to solve the task of trajectory-user link prediction. The framework of model was divided into discriminant module and retrieval module, in which the discriminant module encoded the position information through trajectory embedding, and the improved hierarchical attention network was used to capture the latent correlation between trajectories. The retrieval module used the discriminant module to calculate the similarity score between the known user trajectories and the unknown trajectory. KNN was used as a classifier to link between the unknown trajectory and the user. Experiments were conducted on the dataset based on location-based service (LBS) in a city. The results show that the model in this paper performs well in different user numbers.