Abstract: The object -oriented method solves the problem of segmentation of objects, divides different features into different objects and to a great extent separates the cultivated land, forest land, water, roads, buildings and other typical objects which are inseparable; nevertheless, the object oriented method for features such as shape, texture description is not comprehensive,the amount of information is not enough to support the whole classification and recognition. In this paper, a new method of combining object -oriented and deep learning is proposed, in which the Caffe framework of convolution neural network is used to study the training sample data in depth and,by grasping the texture of different objects and forming deep learning model, guides the classification of objects. The experiment shows that the new method can effectively solve the problem of the low classification accuracy.