Federated　learning　(FL)　has　been　extensively　studied　as　a　means　of　ensuring　data　privacy　while　cooperatively　training　a　global　model　across　decentralized　devices.　Among　various　FL　approaches,　asynchronous　federated　learning　(AFL)　has　distinct　advantages　in　overcoming　the　straggler　problem　via　server-side　aggregation　as　soon　as　it　receives　a　local　model.　However,　AFL　still　faces　several　challenges　in　large-scale　real-world　applications,　such　as　stale　model　problems　and　modality　heterogeneity　across　geographically　distributed　and　industrial　devices　with　different　functions.　In　this　article,　we　propose　a　multimodal　fusion　framework　for　AFL　to　address　the　aforementioned　problems.　Specifically,　a　novel　multilinear　block　fusion　model　is　designed　to　fuse　various　multimodal　information,　which　serves　as　an　enhancement　for　perceiving　and　transmitting　the　important　modality　and　block　during　local　training.　An　adaptive　aggregation　strategy　is　further　developed　to　fully　utilize　heterogeneous　data　by　allowing　the　global　model　to　favor　the　received　local　model　based　on　both　freshness　and　the　importance　of　the　local　data.　Extensive　simulations　with　different　data　distributions　demonstrate　the　superiority　of　the　proposed　framework　in　heterogeneity　scenarios,　which　exhibits　significant　merits　in　the　improvement　of　modality-based　generalization　without　sacrificing　convergence　speed　and　communication　consumption.　　©　2005-2012　IEEE.