Abstract: In order to solve the shortcomings of low accuracy or poor real-time performance of current fatigue detection algorithms, an improved convolution neural network fatigue detection algorithm is proposed. HOG detection algorithm combined with KCF tracking algorithm was used to detect and track the collected faces. The Dlib library was called to extract the key points of the face. A deformable convolution neural network was introduced to identify the extracted eye and mouth states. This algorithm was tested by CEW and YAWDD data set. The accuracy of fatigue detection reaches 94.36%. Experiments show that compared with the current fatigue detection algorithms, the proposed method can detect driver fatigue in real time with high accuracy.