Graph　neural　networks　(GNNs)　are　widely　used　for　analyzing　graph-structural　data　and　solving　graph-related　tasks　due　to　their　powerful　expressiveness.　However,　existing　off-the-shelf　GNN-based　models　usually　consist　of　no　more　than　three　layers.　Deeper　GNNs　usually　suffer　from　severe　performance　degradation　due　to　several　issues　including　the　infamous　“over-smoothing”　issue,　which　restricts　the　further　development　of　GNNs.　In　this　article,　we　investigate　the　over-smoothing　issue　in　deep　GNNs.　We　discover　that　over-smoothing　not　only　results　in　indistinguishable　embeddings　of　graph　nodes,　but　also　alters　and　even　corrupts　their　semantic　structures,　dubbed　semantic　over-smoothing.　Existing　techniques,　e.g.,　graph　normalization,　aim　at　handling　the　former　concern,　but　neglect　the　importance　of　preserving　the　semantic　structures　in　the　spatial　domain,　which　hinders　the　further　improvement　of　model　performance.　To　alleviate　the　concern,　we　propose　a　cluster-keeping　sparse　aggregation　strategy　to　preserve　the　semantic　structure　of　embeddings　in　deep　GNNs　(especially　for　spatial　GNNs).　Particularly,　our　strategy　heuristically　redistributes　the　extent　of　aggregations　for　all　the　nodes　from　layers,　instead　of　aggregating　them　equally,　so　that　it　enables　aggregate　concise　yet　meaningful　information　for　deep　layers.　Without　any　bells　and　whistles,　it　can　be　easily　implemented　as　a　plug-and-play　structure　of　GNNs　via　weighted　residual　connections.　Last,　we　analyze　the　over-smoothing　issue　on　the　GNNs　with　weighted　residual　structures　and　conduct　experiments　to　demonstrate　the　performance　comparable　to　the　state-of-the-arts.　©　2012　IEEE.