In order to enhance the automation and accuracy of road information extraction from the high-resolution remote sensing image (HRI), this paper proposes a HRI road segmentation algorithm, which includes 3 stages, i.e., spectral mergence, edge mergence, and road region extraction based on shape property. The first two stages are actually the image segmentation method based on region growing. Spectral statistic variables, such as average and variance, are considered in the spectral merging criteria to raise segmentation accuracy. A vector gradient method is used to accurately derive edge strength that is critical for edge merging criterion. Spectral and edge mergences are all implemented as global best merge algorithm, so the segmentation result is optimized. On the premise of the complete segmentation of the roads, shape properties can be effectively used to extract roads from HRI. Circularity is adopted to separate roads from non-road regions. Two scenes of OrbView3 multispectral images are used to carry out road extraction experiment. The experimental result shows that the overall accuracy and Kappa coefficient of the method proposed in this paper are above 97% and 0.8, respectively, obviously superior to the result of SVM supervised classification.