Adversarial　training　has　emerged　as　a　straightforward　and　effective　defense　approach　against　adversarial　attacks,　with　ensemble　adversarial　learning　(EAL)　being　a　feasible　branch　to　enhance　the　adversarial　robustness　of　deep　neural　networks　(DNNs).　However,　the　existing　EAL　methods　either　incur　massive　costs　in　multi-model　ensemble　training,　leading　to　low　adaptability,　or　overlook　the　existence　of　gradient　conflicts　in　single-model　self-ensemble　learning,　resulting　in　only　limited　improvement　in　robustness.　To　address　these　issues,　in　this　paper,　we　first　analyze　the　importance　of　weight　state　information　during　network　training,　which　plays　a　key　role　in　ensemble　learning,　especially　in　adversarial　settings.　Then,　we　present　anew　gradient　manipulation　strategy,　it　implements　random　sampling　in　normal　distribution　to　conduct　consensual　gradients　for　alleviating　the　gradient　conflicts.　Based　on　these,　we　propose　a　novel　Weight-wise　Ensemble　Adversarial　Learning　(WEAL),　which　makes　full　use　of　the　states　of　the　weights　and　mitigates　the　conflicts　indifferent　gradients.　It　can　greatly　improve　the　adversarial　robustness　of　the　target　model　within　an　appropriate　consumption　cost.　Extensive　experiments　on　benchmark　datasets　and　models　verify　the　effectiveness　of　the　proposed　WEAL,　e.g.,　in　defending　against　white-box　and　black-box　adversarial　attacks,　compared　to　representative　adversarial　training　methods,　the　adversarial　accuracy　is　increased　by　an　average　of　5.4%　and　4.2%,　and　improving　the　adversarial　accuracy　by　an　average　of　2.8%　and　1.8%　as　compared　to　state-of-the-art　ensemble　adversarial　learning　method.