Abstract: In order to improve the accuracy of multi-class obstacle detection for autonomous vehicles,an improved SegNet neural network algorithm is proposed.Based on SegNet,the algorithm combined residual network and multi-scale fusion algorithm to improve the accuracy of classification.The self-setting contrast normalization algorithm,learning rate adjustment algorithm and class balance algorithm were used to improve the robustness and convergence speed of the network.Through experiments in different scenes,the results show that compared with SegNet neural network,the per pixel accuracy of the improved SegNet neural network is improved from 85％to 97％,and the mean intersection over union is improved from 76％to 90％.