Infrared　and　visible　image　fusion　aims　to　generate　a　comprehensive　image　that　retains　both　salient　targets　of　the　infrared　image　and　texture　details　of　the　visible　image.　However,　existing　methods　overlook　the　differences　in　attention　to　semantic　information　among　different　fused　images.　To　address　this　issue,　we　propose　a　semantic　attention-based　heterogeneous　feature　aggregation　network　for　image　fusion.　The　key　component　of　our　network　is　the　semantic　attention-based　fusion　module,　which　leverages　the　weights　derived　from　semantic　feature　maps　to　dynamically　adjust　the　significance　of　various　semantic　objects　within　the　fusion　feature　maps.　By　using　semantic　weights　as　guidance,　our　fusion　process　concentrates　on　regions　with　crucial　semantics,　resulting　in　a　more　focused　fusion　that　preserves　rich　semantic　information.　Moreover,　we　propose　an　innovative　component　called　the　attentive　dense　block.　This　block　effectively　filters　out　irrelevant　features　during　extraction,　accentuates　essential　features　to　their　maximum　potential,　and　enhances　the　visual　quality　of　the　fused　images.　Importantly,　our　network　demonstrates　strong　generalization　capabilities.　Extensive　experiments　validate　the　superiority　of　our　proposed　network　over　current　state-of-the-art　techniques　in　terms　of　both　visual　appeal　and　semantics-driven　evaluation.