Abstract: Aimed at the problems that the existing image retrieval schemes cannot effectively protect users' privacy, low efficiency and poor retrieval accuracy in practical application, a two-layer retrieval scheme based on information entropy and local sensitive Hash is designed. More representative image features were selected by using the characteristics of image information entropy. Efficient homomorphic encryption algorithm was used to encrypt features. Chaotic encryption algorithm based on information entropy was used to protect image security. Security analysis and experiments show that its performance has reached a good practical level while effectively protecting users' privacy.