Semantic　segmentation　in　high-resolution　aerial　images　is　a　fundamental　and　challenging　task　with　a　wide　range　of　applications.　Although　many　segmentation　methods　with　convolutional　neural　networks　have　achieved　inspiring　results,　it　is　still　difficult　to　distinguish　regions　with　similar　spectral　features　only　using　high-resolution　data.　Besides,　the　traditional　data-independent　upsampling　methods　may　lead　to　suboptimal　results.　This　letter　proposes　a　multisensor　data　fusion　model　(MSDFM).　Following　the　classical　encoder-decoder　structure,　MSDFM　regards　colored　digital　surface　models　(colored-DSMs)　data　as　a　complementary　input　for　further　detailed　feature　extraction.　A　data-dependent　upsampling　(DUpsampling)　method　is　adopted　in　the　decoder　stage　instead　of　the　common　upsampling　approaches　to　improve　the　classification　accuracy　of　pixels　of　the　small　objects.　Extensive　experiments　on　Vaihingen　and　Potsdam　datasets　demonstrate　that　our　proposed　MSDFM　outperforms　most　related　models.　Significantly,　segmentation　performance　for　the　car　category　surpasses　state-of-the-art　methods　over　the　International　Society　of　Photogrammetry　and　Remote　Sensing　(ISPRS)　Vaihingen　dataset.　©　2004-2012　IEEE.