Abstract: Deep learning and related pre-training models have achieved good performance in text classification tasks. The conflict between the generalization requirements of the model and the limited data scale is becoming more and more serious. Gradient descent is used to optimize network parameters, which requires that the network transformation must be continuously differentiable. In addition, the optimization process is easy to be trapped into local minimum values. Based on Granger causality test and optimal transport theory, a performance optimization method for deep learning pre-training models is proposed. The randomization algorithm was combined with the data-driven probability distribution algorithm to generate effective features on a small sample dataset based on the Granger causality test. Based on the optimal transport theory, the optimal combination of effective features was learned to compatible with the instability caused by the transmission mapping between continuous and non-continuous high-dimensional manifold structures. The experimental results show that compared with BERT and TextGCN, the accuracy rates on Chinese and English datasets are both improved.