Abstract: Multispectral （MS） and panchromatic （PAN） images serve as primary data sources for visible-near-infrared optical remote sensing imagery. In a typical land cover classification workflow，the spatial resolution of MS images is generally enhanced using pixel-level fusion methods，followed by image classification. However，the pixel-level fusion process is characterized by considerable time consumption and inconsistency with the optimization objectives of land cover classification，failing to meet the demand for end-to-end remote sensing image classification. To address these challenges，this paper proposed a dual-stream fully convolutional neural network，DSEUNet，which obviates the need for pixel-level fusion. Specifically，two branches were constructed based on the EfficientNet-B3 network to extract features from PAN and MS images，respectively. It was followed by feature-level fusion and decoding，thus outputting the ultimate classification results. Considering that PAN and MS images focus on different features of land cover elements，a spatial attention mechanism was incorporated in the PAN branch to enhance the perception of spatial information，such as details and edges. Moreover，a channel attention mechanism was incorporated in the MS branch to improve the perception of reflectance differences across multiple bands. Experiments on the 10-meter land cover dataset and ablation studies of the network structure demonstrate that the proposed network exhibited higher classification accuracy and faster inference speed. With the same backbone network，DSEUNet outperformed traditional pixel-level fusion-based classification methods，with an increase of 1.62 percentage points in mIoU，1.36 percentage points in mFscore，and 1.49 percentage points in Kappa coefficient，as well as a 17.69% improvement in inference speed.