Pan-sharpening　aims　to　fuse　high　spatial-resolution　panchromatic　images　(PAN)　and　low　spatial-resolution　multispectral　images　(MS)　into　high　spatial-resolution　multispectral　images　(HRMS).We　propose　a　pyramid　hierarchical　multi-spectral　fusion　network,　called　PH-Net　which　can　automatically　fuse　MS　images　and　PAN　images　to　generate　corresponding　HRMS　images.　The　architecture　is　based　on　the　U-Net　network.　First,　a　multi-level　receptive　field　is　realised　by　constructing　an　input　pyramid.　Then,　hierarchical　features　are　extracted　from　the　encoder,　decoder,　and　input　pyramid.　Finally,　the　rich　hierarchical　features　are　used　to　calculate　the　residual　error　between　the　MS　image　and　the　corresponding　HRMS　image.　The　learned　residual　error　is　inserted　into　the　MS　image　to　obtain　the　final　high　spatial-resolution　multispectral　image.　To　demonstrate　the　effectiveness　of　each　component　in　the　network　architecture,　we　conducted　an　ablation　study.　In　addition,　thanks　to　the　design　of　the　multi-layer　architecture,　model　training　does　not　require　a　large　dataset,　which　greatly　improves　the　training　speed　and　significantly　improves　the　generalisability　and　ease　of　deployment　of　this　work　in　the　field　of　remote　sensing　images.　Through　qualitative　and　quantitative　experiments,　we　proved　that　the　proposed　method　is　superior　to　current　advanced　methods.