Abstract: Traditional static graph algorithms struggle to effectively learn node embeddings and the dynamics of network structure evolution. Existing dynamic graph algorithms often rely on node embeddings and fail to handle situations with frequent changes in nodes and edges, both types of algorithms insufficiently consider the nodes' high- order neighborhood. To address these challenges, this paper proposes EvolveStructure. It employed self- aware neighborhood aggregation at each timestep to dynamically learn the high- order neighbors with higher importance, and utilized network structure learning algorithm to capture the dynamic evolution of the graph structure. Results from comparisons on three datasets demonstrate that, compared with the baseline methods, EvolveStructure's performance on node and edge classification tasks is improved by 30% and 16.4% respectively.