Based on fine texture expression of the edge of the detailed information in the digital aerial image，the authors first preprocessed the digital aerial image by edge detection，principal component analysis and the texture filter of second-order probability statistics，secondly obtained the gray image of the contrast texture through the sharpening window of 7×7，then taking the gray image as an independent band，processed a pseudo color composition with the band combination of contrast (R)，the digital aerial image band (G) and the digital aerial image band (B). Finally，multiple segmentation and building extraction in towns and villages based on the pseudo color images were processed. With the digital aerial image acquired in December 2008 from Kangzhuang Town of Yanqing County in Beijing as an example，the authors processed building extraction by using  the method mentioned above. Compared with the object-oriented classification method，it not only highlighted the edges of the buildings but also reduced the redundant segmented objects. Besides，it achieved an effective solution of the shadow of the building and its confusing area，optimized the feature space，and improved the classification accuracy.