Building　change　detection　is　essential　to　many　applications　such　as　monitoring　of　urban　areas,　land　use　management,　and　illegal　building　detection.　This　paper　proposes　a　Siamese　multi-task　neural　network,　labeled　as　SMTNet,　to　detect　building　changes　from　high-resolution　remote　sensing　images.　We　combine　the　advantages　of　the　multi-task　learning　method　and　Siamese　neural　networks　to　improve　the　geometric　accuracies　of　detected　boundaries.　We　applied　the　proposed　method　to　a　very　high-resolution　(VHR)　remote　sensing　dataset　that　is　a　GF2　image-pair　in　Fuzhou　City.　We　also　compared　the　proposed　method　with　two　other　existing　methods,　i.e.,　Obj-SiamNet,　STANet.　Our　results　show　that　the　proposed　method　using　SMTNet　performed　better　than　the　existing　method　from　the　perspective　of　geometric　and　attribute　accuracies.　We　conclude　that　the　proposed　SMTNet　method　has　a　high　potential　for　extracting　changed　boundaries　of　buildings　from　high-resolution　remote　sensing　images.　©　2022　IEEE.