Abstract: Deep learning is one of the important technologies for building extraction from remote sensing images. Aimed at the problems that the current convolutional neural network method has blurred edges, large differences in the extraction results of buildings of different sizes, and large amount of model parameters when extracting buildings, a new method based on attention and multi-scale feature fusion is proposed. The efficient channel attention module was used to enhance the role of important features in network training. The multi-scale feature deep fusion module was embedded to extract and interactively fuse features, and at the same time, the convolution channel pruning idea was used to compress the model. Experiments show that the method have excellent extraction ability, the detail-awareness of the network is stronger, the extracted edges are clearer, the extraction effect is better for buildings with different sizes and irregularities in complex scenes, and the model extraction accuracy and operation efficiency are well balanced.